-- Logs begin at Fri 2022-09-09 10:50:16 UTC, end at Fri 2022-09-09 12:10:26 UTC. -- Sep 09 11:23:56.449389 dv-cinder-856460-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 11:23:56.460614 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 11:23:56 2022] *** Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: nodename: dv-cinder-856460-2-iscsi Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: machine: x86_64 Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: clock source: unix Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: pcre jit disabled Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: detected number of CPU cores: 4 Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: current working directory: / Sep 09 11:23:56.462725 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: detected binary path: /usr/bin/uwsgi-core Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: your processes number limit is 47680 Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: your memory page size is 4096 bytes Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: detected max file descriptor number: 2048 Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: lock engine: pthread robust mutexes Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: thunder lock: enabled Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 11:23:56.463364 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 11:23:56.502181 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: Python main interpreter initialized at 0x560018eac9c0 Sep 09 11:23:56.502181 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: python threads support enabled Sep 09 11:23:56.502181 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: your server socket listen backlog is limited to 100 connections Sep 09 11:23:56.502181 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: your mercy for graceful operations on workers is 90 seconds Sep 09 11:23:56.502513 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 11:23:56.502596 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: *** Operational MODE: preforking *** Sep 09 11:23:56.502876 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: *** uWSGI is running in multiple interpreter mode *** Sep 09 11:23:56.502876 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: spawned uWSGI master process (pid: 81655) Sep 09 11:23:56.503079 dv-cinder-856460-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 11:23:56.504986 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: spawned uWSGI worker 1 (pid: 81664, cores: 1) Sep 09 11:23:56.504986 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: spawned uWSGI worker 2 (pid: 81665, cores: 1) Sep 09 11:23:56.504986 dv-cinder-856460-2-iscsi devstack@keystone.service[81655]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 11:24:00.785377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:24:00.785377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: from cryptography.utils import int_from_bytes Sep 09 11:24:00.786155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:24:00.786155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: from cryptography.utils import int_from_bytes Sep 09 11:24:00.786422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:24:00.786422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: from cryptography.utils import int_from_bytes Sep 09 11:24:00.786877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:24:00.786877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: from cryptography.utils import int_from_bytes Sep 09 11:24:00.836232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:24:00.836404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:24:00.836605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] command line args: [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:24:00.836703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:24:00.836830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:24:00.836915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:24:00.837025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] command line args: [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:24:00.837110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ================================================================================ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:24:00.837283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] admin_token = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.837369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:24:00.837466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ================================================================================ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:24:00.837550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_dir = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.837703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.837788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] admin_token = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.837871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_source = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.837943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_dir = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] debug = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_source = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] debug = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.838984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_config_append = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.839906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_config_append = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_dir = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_dir = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.840905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.841987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.842915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.843039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.843250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.843999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] publish_errors = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.844943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] publish_errors = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.845869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.846023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] standard_threads = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] standard_threads = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_journal = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.849989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_json = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_stderr = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_journal = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_syslog = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_json = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_stderr = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.850771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_syslog = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.850916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:24:00.850986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.851946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.852988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.853975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.854976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855050 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.855936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.856925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.857993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.858943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.859872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.860910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.861929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.862910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.863880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.864976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.865927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.866955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.867938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.868891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.869954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.870899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.871959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.external = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.872861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.external = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.password = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.873981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.token = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.874115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.874234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.874364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.874495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.password = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.token = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.875963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.876901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.877977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.878927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.879951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.880909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881499 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.881961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.882950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.883901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.884830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.885991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.886948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.887891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.888933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.889956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.890135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.890819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.890921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.896938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.897938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.898999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.899943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.900935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.901984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.902974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.903967 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.904997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.905968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.906953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.907874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.908842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.909961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.910932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.911924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.912995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.913932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.914984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915461 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.915947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.916893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.917880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.918909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.919974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.920942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.921935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.922120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.922206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.922342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.922421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.922575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:24:00.924897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:24:00.952584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.952870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.953096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.953272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.953510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.953739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.961411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.962046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.990357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.990586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.990829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:00.991063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.043490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.043662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.092125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.092356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.092691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.092930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.247538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.247749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.247930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.248824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.249040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.271877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.272929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.273019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.273141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.273222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.273355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.284332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.284511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.297497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.297680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.297765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.297910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.297993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.298848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.299000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.299082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:24:01.315368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.315580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:24:01.315924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.316126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:24:01.316401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.316665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.316927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.317188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.317750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.317965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:24:01.318335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.318539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:24:01.318819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.319085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.319344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.319604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.326920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.327202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.327477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.328291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.329891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.330197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.330478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.330769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.335220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.337690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.338089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.338380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.338667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.338954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.339242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.339508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.339781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.340059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.340334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.340604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.340686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.340874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.340954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.341168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.341454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.341741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.342007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.342306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.342584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.342858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.343133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.343460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.363827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.365287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.365572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.366346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.367823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.368112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.370212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.370498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.370784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.372727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.373017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.373302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.377091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.377379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.379666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.379954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.383060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.383357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.383648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.383941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.384222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.384499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.384772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.385044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.385675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.385979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.386300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.386591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.386873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.387154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.387434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.387708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.400814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.401094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.403475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.403760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.404918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.407670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.408208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.408508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.410992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.411297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.412665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.415480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.415801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.418612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.419009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.419311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.419598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.419882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.420160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.420455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.422434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.422740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.423029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.423320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.423605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.423892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.436537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.436828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.437650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.437918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.440064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.440360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.440636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.440905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.444708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.446285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.447726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.447824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.448075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.449262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.451063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.451156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.451345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.451430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.451635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.451920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.452316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.454136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.454429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.454723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.455012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.455417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.463396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.463684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.463969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.464253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.464637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.464934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.465216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.466471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.466760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.467044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.467337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.467726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.468021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.468330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.480367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.483289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.483522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.486697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.486957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.488120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.489697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.489824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.489982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.490402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.491280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.492857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.493145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.493552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.496682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.498162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.499935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.501081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.501379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.501464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.501755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.502033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.504271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.504669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.504959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.505242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.509742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.510033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.510342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.512959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.513255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.513539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.517699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.517985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.518301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.518579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.518862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.519134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.519408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.519691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.519960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.520250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.520521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.520793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.520910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:24:01.521058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.521196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.521326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.521479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.521756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.522032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.522333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.522611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.522904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.523178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.523468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.523751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.524023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.524292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.524566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:24:01.555402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560018eac9c0 pid: 81665 (default app) Sep 09 11:24:01.558627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560018eac9c0 pid: 81664 (default app) Sep 09 11:24:01.558716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2115-1a46-4187-a9fc-16d3b24058da None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:01.558963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2115-1a46-4187-a9fc-16d3b24058da None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:01.559233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2115-1a46-4187-a9fc-16d3b24058da None None] PATH_INFO: `/v3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:01.564006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Fri Sep 9 11:24:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:09.424336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:09.424834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:09.424927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:09.429171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:09.434339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:09.434856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:09.434856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:09.511798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: INFO dbcounter [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] Registered counter for database keystone Sep 09 11:24:09.532571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG oslo_db.sqlalchemy.engines [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81665) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:24:09.538718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writer thread running {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:24:09.716231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG passlib.handlers.bcrypt [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81665) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:24:09.716537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG passlib.handlers.bcrypt [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81665) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:24:09.774466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:09.776048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:09.903871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 09 11:24:10.381986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:10.453595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: INFO dbcounter [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] Registered counter for database keystone Sep 09 11:24:10.473353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG oslo_db.sqlalchemy.engines [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:24:10.479736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writer thread running {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:24:10.606958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91d9a-438f-4042-bc0a-b9ab08c38298 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:10.611296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:10.611909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:10.612190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:10.612462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:10.620576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:10.671627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:24:10.671915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7d6ea-7f04-4c86-899f-88108e2569cd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:10.681614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Fri Sep 9 11:24:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:24:10.688144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:10.706269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bb099-818d-4efd-b61d-d8773041fe1f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:10.710939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:10.712062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:10.712342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:10.712622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] PATH_INFO: `/v3/projects/e4bd719a3a8247ec9289af5d26132989` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:10.714299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] RBAC: Authorizing `identity:get_project(project_id=e4bd719a3a8247ec9289af5d26132989)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:10.769161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:24:10.769505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f0691-43a0-483c-8930-79a096babe97 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:10.774069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Sep 9 11:24:10 2022] GET /identity/v3/projects/e4bd719a3a8247ec9289af5d26132989 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:12.884009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109699-1b60-4d00-9cef-ac4810546c96 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:12.884522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109699-1b60-4d00-9cef-ac4810546c96 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:12.884618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6109699-1b60-4d00-9cef-ac4810546c96 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:12.888650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:12.893980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:12.894425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:12.894563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:12.914678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:12.916185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:12.924553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 11:24:13.189564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6109699-1b60-4d00-9cef-ac4810546c96 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:13.202346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6109699-1b60-4d00-9cef-ac4810546c96 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:13.206476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:13.207059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:13.207340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:13.207633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:13.215085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:13.216822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4312-dfcd-45e0-af2d-49b755ce02a2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:13.225665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Fri Sep 9 11:24:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:24:13.230756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:13.240325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216656d5-6e68-4321-94f8-de5c2331828a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:13.244620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:13.245239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:13.245517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:13.245788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:13.247112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:13.249059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9be-7837-4464-8445-d26ca40ee9bd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:13.250589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Fri Sep 9 11:24:13 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:15.458631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb200f-6ecc-46a5-a3d4-3f4bdcb89b49 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:15.459172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb200f-6ecc-46a5-a3d4-3f4bdcb89b49 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:15.459172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb200f-6ecc-46a5-a3d4-3f4bdcb89b49 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:15.460507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:15.465688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:15.466117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:15.466327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:15.484527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:15.485703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:15.505570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43077fb-f16d-426e-968b-9da571734ef3 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:15.505860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43077fb-f16d-426e-968b-9da571734ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:15.506170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43077fb-f16d-426e-968b-9da571734ef3 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:15.507491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:15.512818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:15.513097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:15.513370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:15.554500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG passlib.handlers.bcrypt [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81664) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:24:15.554893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG passlib.handlers.bcrypt [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81664) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:24:15.567059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:15.603864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:15.605450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:15.704361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:15.834246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:15.857771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:15.935052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ad9c1-f4ce-4bdb-bf0c-e9f63d2e2ed4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:15.968383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:15.968970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:15.969240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:15.969513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:15.970254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:15.980915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:15.989825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:15.990895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:15.991778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:15.996545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.069345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e35477-f9d9-4ab1-b0eb-8d12e29678df None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.071604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.077805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.095002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca50c8-0815-4f8d-b0f2-53fd8c788d34 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.105229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.106100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.106386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.106659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.129085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.129689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.129973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.130299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.130968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:16.131272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.132665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.144316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:16.149779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.172456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.173943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.247408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4500-8458-414c-9634-df1794a58bf2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.269867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:24:16.275450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.278046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.280756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.281384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.281662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.281934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.283093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.284548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.286181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:16 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:16.291912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.306401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.374414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194e6e-6d7a-49f2-bc22-cfc9b961143f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.390453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b58526-7e2b-47b9-a63d-7e1f16e2d922 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.407335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.408187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.408467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.408740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.424379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.424997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.425272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.425539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.426659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.427169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.428098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.428582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.429614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:16.435312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:24:16.446917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:16.447992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:24:16.449283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:24:16.449283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:24:16.449283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:16.449283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:16.453960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:16.508785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.515895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:16.533777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee875-2c9b-48c2-b5f1-fad5bd41d8e1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.552431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674273f-cc61-45cf-a157-d6f2837fa445 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:16.566677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.567281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.567552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.567818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.568537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.domain.id=default, user.id=8ddd128e7075404193b59a38692798c7) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:16.568837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.570304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c25df5-03a5-4a95-a280-92bf1cb2d336 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.577102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Fri Sep 9 11:24:16 2022] GET /identity/v3/role_assignments?user.id=8ddd128e7075404193b59a38692798c7&scope.domain.id=default&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:16.588138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:16.588769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:16.589046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:16.589520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:16.590223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:16.590531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:16.592495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1a7b-24f3-42f8-b64a-52674ed616a9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:16.612480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:19.058823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad5f1-a491-40f0-8bb4-79304cb018a8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.059352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad5f1-a491-40f0-8bb4-79304cb018a8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.059352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad5f1-a491-40f0-8bb4-79304cb018a8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.061249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:19.066516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5659-9550-4f45-8947-7746c8012a70 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.067247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5659-9550-4f45-8947-7746c8012a70 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.067667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5659-9550-4f45-8947-7746c8012a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.103035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ed4f5659-9550-4f45-8947-7746c8012a70 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:19.104507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ed4f5659-9550-4f45-8947-7746c8012a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:19.198218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:19.271010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51aef6-d25c-433e-a434-98b993d5e653 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.271545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51aef6-d25c-433e-a434-98b993d5e653 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.271965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51aef6-d25c-433e-a434-98b993d5e653 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.273519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 13/24] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:19.279712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1b98-6247-4b69-abfd-c3cfba637f4d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.279994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1b98-6247-4b69-abfd-c3cfba637f4d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.280424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1b98-6247-4b69-abfd-c3cfba637f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.307219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dc1b1b98-6247-4b69-abfd-c3cfba637f4d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:19.308625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dc1b1b98-6247-4b69-abfd-c3cfba637f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:19.357744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:19.406502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 12/25] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:19.445085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03a8f-685a-43d3-96b9-444941203684 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.445618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03a8f-685a-43d3-96b9-444941203684 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.446088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03a8f-685a-43d3-96b9-444941203684 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.447572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 14/26] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:19.464275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.464564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.464840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.488098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:19.489274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:19.568400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 13/27] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:19.735549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03a8f-685a-43d3-96b9-444941203684 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:19.776824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:19.882317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4efe0-0da7-4b84-a656-d1a42b207d26 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:19.890372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:19.918254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:19.918729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03a8f-685a-43d3-96b9-444941203684 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:19.918886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:19.919417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:19.919983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:19.930791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:19.932717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:24:19.949803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:19.951300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:24:19.952277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:24:19.952277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:24:19.952277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:19.953731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 14/28] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:19.959691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.042864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.043423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.043704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.043977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.062148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:20.068019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:20.068890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:20.070115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 15/29] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:20.076486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.092270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:20.127824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c524b4-fe52-47e0-9ebf-1e19b28cafd1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:20.203672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.204287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.204571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.204841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.205486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:20.205783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:20.207333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:20.219699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 15/30] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:20.220525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.258823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:20.264789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f33b75-747b-4c76-9d25-ef8e9cb54356 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:20.302129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.302673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.302954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.303229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.303897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:20.304200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:20.305610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:20.315029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 16/31] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:20.315564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.355659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e259d3c-a2ac-4ad3-acf7-41d67dc5d616 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:20.384428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:20.395525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.396146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.396486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.396845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.397440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:20.398911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf858f7d-54d3-485c-aad7-c27e564b75bc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:20.426554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a79f4-8469-40ea-ba1a-ee180f252eee None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:20.464267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.464879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.465160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.465428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.466093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:20.468094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a7011-c1df-45da-ac98-349583e84098 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:20.595014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 16/32] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Sep 9 11:24:20 2022] POST /identity/v3/roles => generated 219 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:20.596397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb679ce-e570-40d6-897b-c1506fd70f9c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.596837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb679ce-e570-40d6-897b-c1506fd70f9c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.597269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb679ce-e570-40d6-897b-c1506fd70f9c None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.598685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 17/33] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:20.599713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb679ce-e570-40d6-897b-c1506fd70f9c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.681307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 17/34] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:24:20 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:20.683123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79430-6917-4bd6-b5ac-d73b20edcb0a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.683682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79430-6917-4bd6-b5ac-d73b20edcb0a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.684126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79430-6917-4bd6-b5ac-d73b20edcb0a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.686336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 18/35] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:20.687802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.688304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.688764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.710001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:20.710311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:20.712225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:20.746994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb679ce-e570-40d6-897b-c1506fd70f9c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:20.805450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:20.806483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.806943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.807371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.809823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:20.811732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2e980-e220-431b-bb8c-fc89cd750525 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:20.813588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 18/36] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:20 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:20.814978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106824-837e-4bba-ba75-92410161f231 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:20.815412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106824-837e-4bba-ba75-92410161f231 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:20.815828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106824-837e-4bba-ba75-92410161f231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:20.819836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 19/37] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:20.821001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:20.838547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-52106824-837e-4bba-ba75-92410161f231 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:20.840372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-52106824-837e-4bba-ba75-92410161f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:20.894818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:20.946264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 19/38] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:20.972803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b6dd-1ca5-4262-a38f-4acbdd559010 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.018808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.019452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.019737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.020011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.021264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.023049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e829f9-4905-4a77-ad2e-3b50172ed150 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.024634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 20/39] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:21.030981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106824-837e-4bba-ba75-92410161f231 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:21.104054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b018e-e11b-470e-b789-064c9cba938c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.104989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b018e-e11b-470e-b789-064c9cba938c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.105472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b018e-e11b-470e-b789-064c9cba938c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.108027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 21/40] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:21.112804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.113364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.113996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.121747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:21.145003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:21.146606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:21.149232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106824-837e-4bba-ba75-92410161f231 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.191088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.191978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.192309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.192600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] PATH_INFO: `/v3/domains/default/users/8ddd128e7075404193b59a38692798c7/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.194633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8ddd128e7075404193b59a38692798c7, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.196742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.230143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 22/41] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:21.231004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:21.280658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:21.306085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 20/42] 10.222.0.60 () {58 vars in 1361 bytes} [Fri Sep 9 11:24:21 2022] PUT /identity/v3/domains/default/users/8ddd128e7075404193b59a38692798c7/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:21.306085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:21.343695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b8c98-d6fc-4020-855c-107bcb49771d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.394373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.395534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.395997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.397040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.397942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.400045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.436466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba767c-dd1b-4bbd-b199-ab2ca7a431d1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.475885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.475885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.475885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.475885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.476310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.477591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.482736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:21.493483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 23/43] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Sep 9 11:24:21 2022] POST /identity/v3/roles => generated 225 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:21.503757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:21.600476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 21/44] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Sep 9 11:24:21 2022] POST /identity/v3/roles => generated 223 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:21.637422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e420dc2-d7b2-46c8-a119-f4d0c09b43df None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.648864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:21.688109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.689151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.689676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.690181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.691624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.693108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bc175-6e0c-489d-9e55-371915f032a4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.695005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 24/45] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:21.701423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:21.818273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03e734-46ca-4c40-a93f-a0f252e0018d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:21.861592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:21.862274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:21.862561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:21.862838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:21.863498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:21.864957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5f221-e7cf-4e8a-96d1-b8004ec75d51 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:21.964504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 22/46] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:24:21 2022] POST /identity/v3/projects => generated 312 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:23.553285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497fc9c-fe03-417b-bc3a-46ce64621c53 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:23.553788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497fc9c-fe03-417b-bc3a-46ce64621c53 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:23.553872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497fc9c-fe03-417b-bc3a-46ce64621c53 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:23.555223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 25/47] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:23.561117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172fa08-870a-47cf-88f0-deeaad29bbb6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:23.561540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172fa08-870a-47cf-88f0-deeaad29bbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:23.561732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172fa08-870a-47cf-88f0-deeaad29bbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:23.580095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e172fa08-870a-47cf-88f0-deeaad29bbb6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:23.581255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e172fa08-870a-47cf-88f0-deeaad29bbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:23.622229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:23.645157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 23/48] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:23.913782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497fc9c-fe03-417b-bc3a-46ce64621c53 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.120589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497fc9c-fe03-417b-bc3a-46ce64621c53 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.130334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974af463-8642-4c01-994c-39745639ff7f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.130861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974af463-8642-4c01-994c-39745639ff7f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.131491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974af463-8642-4c01-994c-39745639ff7f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.133926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 24/49] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:24.140271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.140555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.140837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.160221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.160850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.160994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:24.161124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.161438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.162325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:24.172750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:24.177658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:24.178680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:24.179433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 26/50] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:24.185477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.219597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:24.231440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 25/51] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:24.283617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765e3fe-89e5-4bd1-b75a-a823e71fdffc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.316011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.316616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.316902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.317178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.317843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:24.318182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.319571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.328411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 27/52] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:24.333895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.436317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80b432-6d13-4eba-a55c-387996143f37 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.499605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.547471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:24.573551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.574230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.574510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.575029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.576126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.577650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.579650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 26/53] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:24 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:24.584822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.595478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:24.601084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426f815-ffac-4a1f-9998-2568a6781f1c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.635276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.635904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.636186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.636454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.637551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.639015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.640996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 28/54] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:24.646330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.682878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783fcc-6947-42ea-b44d-cade27efd89c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.717799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.717799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.718029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.718270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.719396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.720826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.722450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 27/55] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:24.728097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:24.745580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe6e26-29a6-436b-a925-233bb58e77cb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.774595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:24.778546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.779226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.779512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.779792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.780445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.781928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703226a-6efb-44c2-9be0-c34d797f71d9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.826853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d4c7-fce3-452d-a8ce-338378910b00 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:24.859569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:24.860167 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:24.860448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:24.860726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:24.861443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.domain.id=default, user.id=8ddd128e7075404193b59a38692798c7) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:24.861750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:24.863188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b039-709f-4edd-b6f8-3e4a0c4358a1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:24.871606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 28/56] 10.222.0.60 () {56 vars in 1322 bytes} [Fri Sep 9 11:24:24 2022] GET /identity/v3/role_assignments?user.id=8ddd128e7075404193b59a38692798c7&scope.domain.id=default&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:24.878152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 29/57] 10.222.0.60 () {60 vars in 1140 bytes} [Fri Sep 9 11:24:24 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:24:27.051927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e692-0bfa-44fd-afa4-6e9605326565 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:27.052445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e692-0bfa-44fd-afa4-6e9605326565 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:27.052552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e692-0bfa-44fd-afa4-6e9605326565 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:27.053811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 29/58] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:27.059134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:27.059500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:27.059709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:27.079172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:27.080370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:27.090675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:27.145276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 30/59] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:27.420925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1e692-0bfa-44fd-afa4-6e9605326565 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:27.517816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1e692-0bfa-44fd-afa4-6e9605326565 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:27.527488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:27.551846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:27.552468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:27.552753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:27.553031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:27.554177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:27.555617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63659-5caa-4580-b3c0-1415ed4dd61a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:27.557178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 30/60] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:27.562594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:27.659061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f915f4-b894-44dd-8357-8a69c76f7533 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:27.691104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:27.691697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:27.691980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:27.692254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:27.692920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:27.694405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ab690-d1da-4ac3-8a2f-fc5c6a49d806 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:27.829810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 31/61] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:27 2022] POST /identity/v3/users => generated 275 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:31.060719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b9fe-9357-42b5-a059-88869a9645e9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.061699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b9fe-9357-42b5-a059-88869a9645e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.062216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b9fe-9357-42b5-a059-88869a9645e9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.063783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 31/62] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:31.069439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf3f-a36a-4f8a-a125-e4bdd97137f1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.070362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf3f-a36a-4f8a-a125-e4bdd97137f1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.070867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72cf3f-a36a-4f8a-a125-e4bdd97137f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.080922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:31.099203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5f72cf3f-a36a-4f8a-a125-e4bdd97137f1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:31.100766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f72cf3f-a36a-4f8a-a125-e4bdd97137f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:31.183434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 32/63] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:31.201392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b7c80-f4f3-4fa3-9764-58abf233d91a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.201966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b7c80-f4f3-4fa3-9764-58abf233d91a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.202477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b7c80-f4f3-4fa3-9764-58abf233d91a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.204146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 32/64] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:31.209584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.210156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.210635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.214731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d5f5-620d-4ce8-a157-8688c0e4dc53 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.215233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d5f5-620d-4ce8-a157-8688c0e4dc53 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.215748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d5f5-620d-4ce8-a157-8688c0e4dc53 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.217232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 33/65] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:31.222136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.222650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.223139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.243058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:31.244319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:31.244619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:31.245976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:31.318077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 34/66] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:31.372508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 33/67] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:31.626497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:31.653842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:31.760260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:31.771301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272aeff-5cd4-4887-8759-528cb4911cbb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:31.780253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:31.806982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076fa7f-1c7b-44aa-9e9e-26269bc431bd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:31.812103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:31.812960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.813829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.814309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.830617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-eb467c20-e3f3-444f-8f5b-56fe913b5c0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:31.838814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:31.839763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:31.841092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 35/68] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:31.842041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3af1b-d475-4ad6-8aee-1e7d56e7b9e2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.842515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3af1b-d475-4ad6-8aee-1e7d56e7b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.842940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3af1b-d475-4ad6-8aee-1e7d56e7b9e2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.844448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 36/69] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:31.845596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3af1b-d475-4ad6-8aee-1e7d56e7b9e2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:31.857223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:31.857867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.858180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.858458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:31.869518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:31.875441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:31.876425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 34/70] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:31.877954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:31.963332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3af1b-d475-4ad6-8aee-1e7d56e7b9e2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:31.980918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55f0-9a1e-4b53-851b-75947bd46ae0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:31.995966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:31.996772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:31.997053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:31.997606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.002476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.008716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:32.013725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.014361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.018164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.018164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2dddc455-89f1-467e-a34a-6e1b029c76b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:32.018333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:32.019116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:32.019863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:32.020435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:32.020435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.020547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 37/71] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:32.020608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.020676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4de28-6ac7-40b7-9b30-84e71dcb16d3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.020676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4de28-6ac7-40b7-9b30-84e71dcb16d3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.020676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4de28-6ac7-40b7-9b30-84e71dcb16d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.028832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 35/72] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.029653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.036466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7de4de28-6ac7-40b7-9b30-84e71dcb16d3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:32.037675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7de4de28-6ac7-40b7-9b30-84e71dcb16d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:32.057474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.103796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 38/73] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:32.105143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67ffe-bbca-4335-a708-78d8f0414a0a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.105440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67ffe-bbca-4335-a708-78d8f0414a0a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.105719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67ffe-bbca-4335-a708-78d8f0414a0a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.107493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 39/74] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:32.108196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67ffe-bbca-4335-a708-78d8f0414a0a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.139715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63571d4-04fb-45f9-9597-3d88b094eb46 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.177395 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.178368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.178640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.179387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:32.179916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.181494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.191869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 36/75] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.192920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.229248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67ffe-bbca-4335-a708-78d8f0414a0a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.243867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.265070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.265836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.266332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.266773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.267594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:32.268087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.269841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ac424-8ce2-4c5d-ae4f-38a825d9a3bb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.287720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 40/76] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.289304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c7040-df53-458f-801f-9712398a147e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.289765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c7040-df53-458f-801f-9712398a147e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.290259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c7040-df53-458f-801f-9712398a147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.304192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c625a-2f9c-48de-892b-49a0e1232624 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.308678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f28c7040-df53-458f-801f-9712398a147e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:32.310253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f28c7040-df53-458f-801f-9712398a147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:32.323849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.344162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.345660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.346193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.346642 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.348078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.349893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.352566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 37/77] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.353588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.391364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 41/78] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:32.392240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c7040-df53-458f-801f-9712398a147e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.486794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdd903-196e-4220-9210-cf4284df896e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.503501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.509080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c7040-df53-458f-801f-9712398a147e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.532716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.533776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.533776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.533965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.549999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.551508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.553165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 38/79] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.553939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.561371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.562348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.562637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.562911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.564083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.565530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.571817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 42/80] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.571817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.589623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.665080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26300-c9a5-4a6a-a387-d3836582c976 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.677138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb08e3e-6118-442a-961a-d2f4abd5865c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.698368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.698975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.699526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.700463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.711086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.711716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.711998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.712272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.713400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.714875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.716477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 43/81] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:32.717247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.741719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.768384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.769872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.801164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.816650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c5d25-20f7-48f0-9999-8b6f5be47146 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.849966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:32.850621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:32.850906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:32.851186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:32.866521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 39/82] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:32.867359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:32.922318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:32.923695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:32.937398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:32.965681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5c148-1118-433b-8ffa-67a257cda80f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:32.999737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.000336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.000619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.000904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.006702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.025613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 44/83] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:33.026778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.068811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.070309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.127035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b65e3d-3036-4e38-b8e6-f5ba45caf5da None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.139329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.160636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.161266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.161547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.161983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.169766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 40/84] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:33.170447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.177301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:33.182232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:33.183089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:33.183855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 45/85] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:33.184509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.230335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.270475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a772e-c68e-447d-9cd8-bb4c0e8c797b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.282664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323853f-2391-4f1c-87ca-6181cac2593e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.303322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.303931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.304218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.304494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.305151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.306632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.315473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.316101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.316379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.316665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.317785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.319270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.320885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 46/86] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.326254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.400307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 41/87] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:33 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:33.401177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.420923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d3cb-b322-498b-8a93-3a56b1f02367 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.438199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.455647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.464616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.465306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.466254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.466593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.468007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.469597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.471835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 47/88] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.472905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.544117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf97c-da03-4681-a630-de1a69d3b53e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.582190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.583554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.583554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.583554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.584562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.586034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.587717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 42/89] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.588478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.593332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4183-4a85-4300-84c8-d6e149ecd8a6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.634627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.634627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.634627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.634627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.635537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:33.635852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.638133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.647602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 48/90] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.649156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.698887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.708096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee348f6-6ce1-433d-b4b4-46b13626b636 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.711069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:33.759812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.760778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.761068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.761340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.762702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=8ddd128e7075404193b59a38692798c7) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:33.763951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.765939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.773126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 43/91] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/role_assignments?user.id=8ddd128e7075404193b59a38692798c7&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.774878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.798153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c124f2-0006-4c7b-b4b7-558ec9fa29a5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.863586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.864214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.864495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.864770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.865518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:33.865831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.869630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:33.882891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 49/92] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=d3096edb08e84cd098dda68154bffd4b => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:33.883855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:33.919896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f943-3d50-4f40-b37c-10091b2e4156 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:33.994928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:33.995493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:33.995779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:33.996055 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:33.997381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:33.997694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:33.999204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:34.010892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 44/93] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=28d83235f01b49b98d223883c8317b87 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:34.105125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df56f3-d7e2-4dce-952c-7b9685d54d2e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:34.116975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:34.159421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:34.160868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:34.161342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:34.161872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:34.163294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:34.164934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:34.166741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 50/94] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:33 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:34.172988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:34.211228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:34.316703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78175e2-bb54-46a5-ad57-95c51b9338ce None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:34.365498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:34.366546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:34.367043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:34.367539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:34.477577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:34.479077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:34.511843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:34.621877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 45/95] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:34.627391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:34.790074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576776-70de-4a2b-9ef2-805e44d2db77 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:34.839916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:34.841123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:34.841397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:34.841666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] PATH_INFO: `/v3/projects/4975e65cacb64c73bc0d46ee47678e93` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:34.846209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] RBAC: Authorizing `identity:get_project(project_id=4975e65cacb64c73bc0d46ee47678e93)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:34.847651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d64f55-ed47-4bed-9b3c-9edc0170af10 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:34.849391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 51/96] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:34 2022] GET /identity/v3/projects/4975e65cacb64c73bc0d46ee47678e93 => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:34.855575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:34.984472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2d92c-5463-4530-a9da-076bde509a9f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:35.013440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:35.031278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:35.031895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:35.032172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:35.032444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:35.033284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=4975e65cacb64c73bc0d46ee47678e93, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:35.033592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:35.035104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859e8a8-b780-46e5-8bab-eb644235c64c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:35.040856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 46/97] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:34 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=4975e65cacb64c73bc0d46ee47678e93&role.id=d3096edb08e84cd098dda68154bffd4b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:36.594469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc590af-18cd-4d64-a8eb-d9106ad50fcb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.594967 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc590af-18cd-4d64-a8eb-d9106ad50fcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.595041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc590af-18cd-4d64-a8eb-d9106ad50fcb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.596330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 52/98] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:36.611528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5713e1-9544-4bb3-a9d8-ab8b7b12d61f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.611992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5713e1-9544-4bb3-a9d8-ab8b7b12d61f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.612178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5713e1-9544-4bb3-a9d8-ab8b7b12d61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.630409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8b5713e1-9544-4bb3-a9d8-ab8b7b12d61f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:36.631537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8b5713e1-9544-4bb3-a9d8-ab8b7b12d61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:36.703493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 47/99] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:36.929662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c239-f795-4501-940c-1de10fc22a02 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.929992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c239-f795-4501-940c-1de10fc22a02 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.930193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86c239-f795-4501-940c-1de10fc22a02 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.931497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 53/100] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:36.937676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ebbbc-c550-423b-885b-7488bbf1fd23 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.938264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ebbbc-c550-423b-885b-7488bbf1fd23 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.938731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ebbbc-c550-423b-885b-7488bbf1fd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.942331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1251cd-c6bc-4338-b3f6-cd0094e58b1f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.943349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1251cd-c6bc-4338-b3f6-cd0094e58b1f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.944827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1251cd-c6bc-4338-b3f6-cd0094e58b1f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.949168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 48/101] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:36 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:36.950601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813304e-a672-4a76-8372-202287bba81c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.951090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813304e-a672-4a76-8372-202287bba81c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.951560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813304e-a672-4a76-8372-202287bba81c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.952976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 49/102] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:36.958334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:36.958856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:36.959322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:36.969972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-403ebbbc-c550-423b-885b-7488bbf1fd23 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:36.972446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-403ebbbc-c550-423b-885b-7488bbf1fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:36.991032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:36.992292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:37.019431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.054683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 54/103] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:37.056239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.056716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.057182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.079020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:37.081269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:37.116322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 50/104] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:37.117476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.157308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 55/105] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:37.201556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.271885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d1590-a438-4960-ae2a-4c2b5a6ea2e4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.316095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.316734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.317018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.317295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.330828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:37.335689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:37.336539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:37.337303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 51/106] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:37.342857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.463553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fccc69-f0c9-4ec5-9d5f-816256d22a07 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.465455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.477334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.501668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.502312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.502592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.502853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.503636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:37.503943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:37.505310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:37.514843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 56/107] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:37.520161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.655105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb920bac-bc3d-48a2-9261-776c0a1099b1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.656408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96de21-16c0-437a-ac14-b21e14d7ff71 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.665708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.691393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.691564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.692075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.692177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.692531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.692627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.692823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.692926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] PATH_INFO: `/v3/users/10d2119ed8ff421196c83d04b8633795` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.704748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:37.708435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] RBAC: Authorizing `identity:get_user(user_id=10d2119ed8ff421196c83d04b8633795)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:37.709673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:37.710550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:37.711332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:37.711909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25e71a-64c6-4ed8-99c4-ba19990d500c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:37.711974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 52/108] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:37.711974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.712086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 57/109] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/users/10d2119ed8ff421196c83d04b8633795 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:37.715970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892e9c-50ab-4842-9b30-f377cb0a9a91 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.716477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892e9c-50ab-4842-9b30-f377cb0a9a91 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.717010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b892e9c-50ab-4842-9b30-f377cb0a9a91 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.718531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 58/110] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:37.722848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b892e9c-50ab-4842-9b30-f377cb0a9a91 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.747410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.828249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f7fbd-85f9-4c81-a7f0-5be958491e96 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.828796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b892e9c-50ab-4842-9b30-f377cb0a9a91 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.864655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.864848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:37.865273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.865480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:37.865574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.865729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:37.865819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.865995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:37.867153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:37.868698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:37.870306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 59/111] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:37.874237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.881173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:37.886601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:37.887548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:37.888581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 53/112] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:37.889761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:37.917437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.976879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:37.987374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48cf4-b7af-44e1-bf7e-487202e5390c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:37.997652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fb277-d0e5-4904-b255-d6a270f33e58 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.022273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.023106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.023792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.024145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.034841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.035633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.035969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.036303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.037178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:38.037535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.039096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb88f19-093e-47fc-924a-8856403d7c13 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.052431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 54/113] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.053708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.054087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.054441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.072252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:38.073436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:38.097492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.099190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.144232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 55/114] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:38.145224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.176630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.219107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.234312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 60/115] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:38.235514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.281656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4a064-f671-487d-a5d8-14c617c9f059 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.317791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.318441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.318821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.319220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.319981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.321541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.345217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2e746-c280-4d37-a90e-be5db536fbe5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.380873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.381520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.381845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.382134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.382811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:38.383121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.384488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.393121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 61/116] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.393905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.408954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 56/117] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:24:38 2022] POST /identity/v3/projects => generated 302 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:38.410078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.471620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.489665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.523518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746292e-bd0a-490c-8474-fe817c8a1d70 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.523980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bce09-5513-49cb-a204-0f11eeb92a8f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.561953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.562805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.563164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.563514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.563754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.564460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.564914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.565017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.565366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.566743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.567829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ee483-12a5-41dd-a000-c948efa49983 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.568326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.569889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 57/118] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.569991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 62/119] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.571308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb97fc-163e-4ccb-abe9-7b986098e2cb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.571598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb97fc-163e-4ccb-abe9-7b986098e2cb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.571876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb97fc-163e-4ccb-abe9-7b986098e2cb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.577770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 58/120] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:38.577770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb97fc-163e-4ccb-abe9-7b986098e2cb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.577955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.682047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40df2e-b32b-4246-a9e9-9502a2336d92 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.699239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb97fc-163e-4ccb-abe9-7b986098e2cb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.705278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.719624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.720256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.720568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.720822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.721959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.723472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.725200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 63/121] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.725784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.742868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.743431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.743713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.743988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.752135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:38.755625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:38.760839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:38.766236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 59/122] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:38.767043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.864332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502f06-0827-4cbe-864f-4a1903cf6a04 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.866590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317edb-f619-4181-bbef-dce51c42d413 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:38.900978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.901586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.901871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.902203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.902515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:38.902935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=10d2119ed8ff421196c83d04b8633795) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:38.903150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.903240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.903433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.903709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.904705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.904861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:38.906411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eb81c-c51c-42ff-9023-f608d4834b73 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:38.907943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 64/123] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.909045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:38.909331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:38.909604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:38.915564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 60/124] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/role_assignments?user.id=10d2119ed8ff421196c83d04b8633795&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:38.916232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:38.944410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:38.945650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:39.001484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.017176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.035281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 65/125] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:39.044291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.075110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ccea8-2897-486a-9ad5-1b3dd440b921 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.150513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.152262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.154168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.154685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.157057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:39.159460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:39.163757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 61/126] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:38 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:39.167995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.213907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec14a7-787a-4c3a-940c-c2ad340f717c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.280125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.281652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.282334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.283802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.284698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:39.285867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:39.287452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:39.302711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 66/127] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:39.304537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.348676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575f66c-9c31-4b7e-a003-203051025eca None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.369615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.392123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.398793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.399650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.400130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.401010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/59019ff482684682b14c5cdbb421e5b2/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.403653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, user_id=59019ff482684682b14c5cdbb421e5b2, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:39.405629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:39.427217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cdd0-3ae2-42da-bc4d-fe3225b3be04 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.463313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.463996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.464279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.464610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/8ddd128e7075404193b59a38692798c7/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.466592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, user_id=8ddd128e7075404193b59a38692798c7, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:39.468146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:39.486539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 62/128] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:39 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/59019ff482684682b14c5cdbb421e5b2/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:39.487160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.602377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 67/129] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:39 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/8ddd128e7075404193b59a38692798c7/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:39.603161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.631848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192f61a-7699-446a-929a-5373c7af0fa6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.724920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.725596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.725877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.726173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.752930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:39.758399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:39.759334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:39.760128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 63/130] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:39.769015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.794629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f1c43-d7d5-4942-a4a9-16d57df00839 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:39.804853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.835110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:39.836167 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:39.836456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:39.836741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:39.838006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:39.840404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:39.842568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 68/131] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:39 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:39.851157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:39.881279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:39.966717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f593d-db35-4403-b978-e7d937422e9d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.001873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78d39-86f4-4a86-95de-39149fc9105c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.018926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.019552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.019884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.020220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.020907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:40.021215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.022702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.032280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 64/132] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:40.037693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:40.070185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.070826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.071106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.071384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.073222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.075013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.078218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 69/133] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:39 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:40.082603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:40.167287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:40.205718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3fd-0581-47df-94fa-75ea14325ea3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.227046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:40.236345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc437bd-43f8-4aac-ac2b-91837b545bab None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.276161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.276837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.277154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.277429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.278960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.287283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.289210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 65/134] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:40 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:40.298754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.299759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.300268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.300735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/59019ff482684682b14c5cdbb421e5b2/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.304253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:40.304942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, user_id=59019ff482684682b14c5cdbb421e5b2, role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.306752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.395129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 70/135] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:40 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/59019ff482684682b14c5cdbb421e5b2/roles/28d83235f01b49b98d223883c8317b87 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:24:40.472908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780ebad-3825-4ee9-b54f-e04a84e55b40 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.514732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.515378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.515664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.515945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] PATH_INFO: `/v3/projects/4975e65cacb64c73bc0d46ee47678e93` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.522806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] RBAC: Authorizing `identity:get_project(project_id=4975e65cacb64c73bc0d46ee47678e93)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.524340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42c6b2-8e27-41f6-b374-863d4f235e44 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.526149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 66/136] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:40 2022] GET /identity/v3/projects/4975e65cacb64c73bc0d46ee47678e93 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:40.532799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:40.675604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9db4a-a2ee-4aaf-b4ad-cda0c10f923a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:40.679439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:40.732618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:40.733234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.733525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.733803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] PATH_INFO: `/v3/projects/4975e65cacb64c73bc0d46ee47678e93/users/59019ff482684682b14c5cdbb421e5b2/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.735844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] RBAC: Authorizing `identity:create_grant(project_id=4975e65cacb64c73bc0d46ee47678e93, user_id=59019ff482684682b14c5cdbb421e5b2, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:40.737294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c6d5f-3ff3-4e02-8d8b-7c7e45254944 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:40.814340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 71/137] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:40 2022] PUT /identity/v3/projects/4975e65cacb64c73bc0d46ee47678e93/users/59019ff482684682b14c5cdbb421e5b2/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:40.919253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8beb-c4ca-4217-86c2-ea992d1f7e68 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.919755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8beb-c4ca-4217-86c2-ea992d1f7e68 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.919828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8beb-c4ca-4217-86c2-ea992d1f7e68 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.921272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 67/138] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:40.927270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:40.927578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:40.927836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:40.959289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:40.960544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:41.049610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 72/139] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:41.339220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8beb-c4ca-4217-86c2-ea992d1f7e68 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:41.405772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:41.466529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8beb-c4ca-4217-86c2-ea992d1f7e68 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:41.501944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:41.502624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:41.502905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:41.503179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:41.504295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:41.505803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d422d-9edf-498e-a76e-086b9b28a316 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:41.507712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 68/140] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:41.513358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:41.628865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:41.636840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c1860-4b04-4632-a587-ebf910d38a86 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:41.682486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:41.683158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:41.683452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:41.683726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:41.684391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:41.685823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:41.828617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df497a4-ce92-47e0-89c6-b59e9b9f7972 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:41.829037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df497a4-ce92-47e0-89c6-b59e9b9f7972 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:41.829212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df497a4-ce92-47e0-89c6-b59e9b9f7972 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:41.830554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 69/141] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:41.835880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:41.836183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:41.836438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:41.856476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 73/142] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:41 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:41.867014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:41.868237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:41.969065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 70/143] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:42.531585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:42.654381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3cb2-659d-4ef0-a45b-71bcb3e1ea96 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:42.699754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:42.700401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:42.700679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:42.700950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:42.719616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:42.724505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:42.727721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:42.728480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 74/144] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:42.734519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:42.794922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:42.858002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c4bd7-1072-4fad-81c8-9ed78cddd99a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:42.898967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:42.899820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:42.900099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:42.900370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:42.901050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:42.901470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:42.903505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9ff01-e620-4b69-a7d8-9760aed28ff6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:42.912683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 71/145] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:42.918680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:42.931831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f4d3-db51-4d0f-8a5c-d8ddfa23009f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:42.932107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f4d3-db51-4d0f-8a5c-d8ddfa23009f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:42.932385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f4d3-db51-4d0f-8a5c-d8ddfa23009f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:42.934535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 72/146] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:42.936214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:42.944224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:42.947167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:42.947167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:42.982897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:42.984107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:43.088030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 73/147] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:43.111820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448ebf5-5a97-417e-b7ba-b00b6a8b97b2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:43.172244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:43.172807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.173087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.173360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] PATH_INFO: `/v3/users/10d2119ed8ff421196c83d04b8633795` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.178767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] RBAC: Authorizing `identity:get_user(user_id=10d2119ed8ff421196c83d04b8633795)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:43.180239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da88980-3f87-4df1-a046-a4cfbd4cdc53 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:43.182026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 75/148] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:42 2022] GET /identity/v3/users/10d2119ed8ff421196c83d04b8633795 => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:43.188188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:43.239789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:43.268273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e571-93ec-4030-be2a-4017ad7bb4ef None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.268605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e571-93ec-4030-be2a-4017ad7bb4ef None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.268826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572e571-93ec-4030-be2a-4017ad7bb4ef None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.270356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 76/149] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:43.275651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.275933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.276208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.312227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:43.313470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:43.358254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e166e4d-5a27-4076-acc9-a236c6704daf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:43.446247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:43.446812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.447090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.447362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.449952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:43.451434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:43.454119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 74/150] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:43 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:43.457542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 77/151] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:43.459561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:43.589544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0816d9-8311-4cc4-8b8c-44237d1cf9ed None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:43.600393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:43.608233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:43.619653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:43.662902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:43.663591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.663866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.664137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/10d2119ed8ff421196c83d04b8633795/roles/2aa07c7012a14708a05efb12b4f81849` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.666274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, user_id=10d2119ed8ff421196c83d04b8633795, role_id=2aa07c7012a14708a05efb12b4f81849)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:43.667867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59be59-740c-4c3c-9567-8522c4846826 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:43.746005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081af4-2aa6-41fe-8f74-895feb0af315 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:43.753751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 75/152] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:43 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/users/10d2119ed8ff421196c83d04b8633795/roles/2aa07c7012a14708a05efb12b4f81849 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:24:43.761473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093917-1961-4aa4-9acf-748cc8743dc9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.762818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093917-1961-4aa4-9acf-748cc8743dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.762818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093917-1961-4aa4-9acf-748cc8743dc9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.763765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 76/153] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:43.773302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.773614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.773851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.816903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:43.818136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:43.836871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:43.837531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:43.837813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:43.838106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:43.866816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:43.872706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:43.873592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:43.874417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:43.876805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 78/154] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:43.877813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:43.923603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 77/155] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:43.924487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.067624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.076076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.080880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4762e4-7c24-4d1d-b419-2864d63548b3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.119504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821bea2-ddef-40cf-bbec-ca0f48e74d99 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.121377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.122005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.122323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.122590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.123266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:44.123562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:44.124951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:44.138431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 78/156] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:44.150331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.165251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.166231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.166709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.167174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.180836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:44.186433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:44.187349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:44.188166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:44.190509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 79/157] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:44.196818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.357331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522de46-1eef-47d6-8506-b9931d562745 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.404073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aca0b-4e04-40db-bd8e-f7e27ce132d2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.404698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.417778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.418865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.418865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.419066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.419737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:44.420033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:44.421417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:44.434317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 80/158] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:44.440912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.476142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.476785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.477061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.477329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] PATH_INFO: `/v3/users/8ddd128e7075404193b59a38692798c7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.482777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] RBAC: Authorizing `identity:get_user(user_id=8ddd128e7075404193b59a38692798c7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:44.484247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:44.486192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 79/159] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:44 2022] GET /identity/v3/users/8ddd128e7075404193b59a38692798c7 => generated 247 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:44.491763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.548658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.586322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938be3-2061-480c-9b40-0c432bb8b196 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.623065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.623709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.624019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.624294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.625583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:44.627289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b22b57-9caa-475c-a3d7-1d8ffe60ffca None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:44.629374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 81/160] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:44 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:44.630649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4130b-a508-410a-9937-0c583a67dbae None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.630925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4130b-a508-410a-9937-0c583a67dbae None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.631196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4130b-a508-410a-9937-0c583a67dbae None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.632633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 82/161] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:44.639531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.639820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.640052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.667822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:44.669019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:44.699486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2121fb-1143-4d96-a9ab-e691fddc941a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.758438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.772984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 83/162] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:44.773916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:44.775346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.775973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.776249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.776568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.879088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32bf53-f473-4a2a-a321-54121181216a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:44.898538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:44.900007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:44.914647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:44.915273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:44.915651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:44.915931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:44.943200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.978453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:44.998711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.000461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.058445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 80/163] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:24:45.058773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.153356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 84/164] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:45.154892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.206526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20b9b7-b775-4534-ba5a-a9071c953a36 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.243294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:45.254078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.254078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.254078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.254078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.258498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:45.267805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:45.279018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:45.283883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 81/165] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:45.284668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.285557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09aa4b-6c4a-4296-b159-63881bdd652e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.362730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.363489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.363773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.364134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.365313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.366872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.368519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 85/166] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:45.369400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.403593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea921a-9da7-4017-b950-5cede28af82a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.470794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.471445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.471722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.471996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.473167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.474678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.476527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 82/167] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:45.477078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.556953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:45.580956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023267e8-0206-48f8-bf28-dd6be7ceb907 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.600388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ffaff-525a-4d82-8f79-d10a8ad35d13 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.639984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.640591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.640870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.641141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.641867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=8ddd128e7075404193b59a38692798c7) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:45.642210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.643677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.646659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.647269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.647543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.647809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.648474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:45.648771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.650137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.660071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 83/168] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/role_assignments?user.id=8ddd128e7075404193b59a38692798c7&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:45.660845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.666332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 86/169] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:45.667264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.718450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:45.882811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362413f-1b68-4f37-bd28-3082310ca7fa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.897793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bb412-aca5-4014-bf55-3472f5a0640b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:45.933770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:45.955327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.955954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.956230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.956496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.957238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:45.957550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:45.958430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:45.959040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:45.959152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:45.959361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:45.959630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:45.976483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 87/170] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=d3096edb08e84cd098dda68154bffd4b => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:45.977356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:45.988758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:45.994349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:45.995783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:45.997260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 84/171] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:46.002280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:46.179480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:46.212399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307ecad-0566-45f2-91c2-108fc0eb909d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:46.228983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5b065-ae55-435c-83b5-aa2669b7c4e5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:46.290033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:46.290671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.290948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.291219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.291747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:46.291959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:46.292732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:46.293522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.294004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.294499 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.298297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:46.299048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:46.301668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:46.303757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 88/172] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:45 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:46.309388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 85/173] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:46.309688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:46.314515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:46.416504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:46.504211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348bb61-d5b1-4183-baa9-b6dd69316eb1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:46.550032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:46.550730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.550975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.551245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] PATH_INFO: `/v3/users/59019ff482684682b14c5cdbb421e5b2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.552723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=59019ff482684682b14c5cdbb421e5b2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:46.554384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748369-9655-4531-ab2d-25dcb0f4a5c2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:46.556825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 86/174] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:46 2022] GET /identity/v3/users/59019ff482684682b14c5cdbb421e5b2 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:46.558421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2be4-d972-4270-a614-305baba69be5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.558855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2be4-d972-4270-a614-305baba69be5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.559246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d2be4-d972-4270-a614-305baba69be5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.560467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 87/175] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:46.561003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a46d-41e7-48c6-aae1-975f23fa0443 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:46.562938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2be4-d972-4270-a614-305baba69be5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:46.603321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:46.604092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.604556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.604927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.626750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:46.684495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:46.685931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089aa000-cb4f-492d-9b8a-148e62e02bd2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:46.775360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d2be4-d972-4270-a614-305baba69be5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:46.797922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:46.842493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:46.843222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.843557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.843877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.877542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 89/176] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:24:46.884654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d944e4-ba56-4997-b013-88654171538d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:46.884941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d944e4-ba56-4997-b013-88654171538d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:46.885213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d944e4-ba56-4997-b013-88654171538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:46.912416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-78d944e4-ba56-4997-b013-88654171538d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:46.913718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-78d944e4-ba56-4997-b013-88654171538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:46.971543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:46.973758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:46.981394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.022038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 90/177] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:47.025463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d944e4-ba56-4997-b013-88654171538d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.115233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 88/178] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:47.121157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.132723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.163460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d944e4-ba56-4997-b013-88654171538d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:47.213719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:47.215882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.216900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.217345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.218681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:47.220985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:47.223296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 91/179] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:47.229868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.237207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb8bf-6d9a-4583-8361-ed45316bbc78 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:47.275556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:47.276282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.277210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.277574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] PATH_INFO: `/v3/projects/4975e65cacb64c73bc0d46ee47678e93` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.278824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=4975e65cacb64c73bc0d46ee47678e93)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:47.280353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:47.282408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 89/180] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/projects/4975e65cacb64c73bc0d46ee47678e93 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:47.288235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.297207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.380325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081e64b-5579-418c-9f97-bc63f35e1a4c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:47.386419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.424377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:47.424983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.425258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.425526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.429532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:47.430364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:47.431799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d794234-a32c-4d74-aba5-83d862bfb105 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:47.434514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5a74-379c-41ca-a0e5-438b3ff30cb9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:47.446852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 92/181] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=28d83235f01b49b98d223883c8317b87 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:47.490958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:47.491593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.491868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.492137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.492879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=4975e65cacb64c73bc0d46ee47678e93, user.id=59019ff482684682b14c5cdbb421e5b2) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:47.493185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:47.494688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a550-eceb-441a-9010-ca2e9308b33f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:47.501299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 90/182] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/role_assignments?user.id=59019ff482684682b14c5cdbb421e5b2&scope.project.id=4975e65cacb64c73bc0d46ee47678e93&role.id=d3096edb08e84cd098dda68154bffd4b => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:47.518152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde206-6c81-4a6e-b7b7-ec3e92cf74af None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.518480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde206-6c81-4a6e-b7b7-ec3e92cf74af None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.518700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bde206-6c81-4a6e-b7b7-ec3e92cf74af None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.520241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 93/183] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:47.526036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ab9e-099b-45e1-8386-c7e18a0dcae4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.526871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ab9e-099b-45e1-8386-c7e18a0dcae4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.527114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ab9e-099b-45e1-8386-c7e18a0dcae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.551501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7025ab9e-099b-45e1-8386-c7e18a0dcae4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:47.552705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7025ab9e-099b-45e1-8386-c7e18a0dcae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:47.570382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bde206-6c81-4a6e-b7b7-ec3e92cf74af None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.669192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 91/184] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:47.772443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cb5f-c490-4b99-8b8a-039e98f906d1 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.772789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cb5f-c490-4b99-8b8a-039e98f906d1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.772993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cb5f-c490-4b99-8b8a-039e98f906d1 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.774576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 92/185] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:47.782870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.783216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.783586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.803069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:47.803832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bde206-6c81-4a6e-b7b7-ec3e92cf74af None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:47.804272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:47.829805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.857916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:47.858589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.858869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.859142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.875725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-86501ead-6753-419c-973f-5e7b375df10f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:47.880908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:47.881767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:47.882629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 94/186] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:47.883548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697a1e-5c09-4e13-919c-7cc8fee75d56 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.883826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697a1e-5c09-4e13-919c-7cc8fee75d56 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.884098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697a1e-5c09-4e13-919c-7cc8fee75d56 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.889253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 95/187] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:47.894189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 93/188] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:47.895165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:47.895441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:47.895720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:47.901528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:47.904200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:47.919178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:47.920385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:47.995261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 96/189] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:48.020418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce871c4-c675-46b8-81b0-2011790243f4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.040831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.064807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.065973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.066274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.066545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.067523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:48.067826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.069452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.082109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 94/190] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:48.090512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.160017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc00ef-edcd-481b-9e88-b69764339a5e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.161632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.167806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.196586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.197224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.197500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.197769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.208573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd087a-b8eb-4cf9-b7e6-f34eebf6689d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.208916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:48.214425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:48.216224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 97/191] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:48.217476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.253276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.253903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.254218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.254498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.272783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.274887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.277176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 95/192] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.278108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.323207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6840a8-4848-415a-aee0-331281d76f02 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.358558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.359296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.359576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.359848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.370735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:48.375607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:48.376490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:48.377257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 98/193] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:48.377900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.381156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73770d4-650c-4f3c-a431-b3627419540e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.405071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.405382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.419635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.420569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.421310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.422029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.422923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:48.423105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.424481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.432510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 96/194] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.433579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.481209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea72f08-c655-4140-9617-bd7f7d43013c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.513795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.514425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.514701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.514970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.533465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e3893-c9a8-4304-8261-adfbf5692c46 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.569091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.569928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.570412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.570688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.571875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.573667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.575231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 97/195] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.575992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.581897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.583371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.606914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.627414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.674450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bfc45-8552-48d4-9e97-5e74beb847b5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.681468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 99/196] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:48.682358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.709087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.709687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.709963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.710254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.710931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:48.715758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.715758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.720737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 98/197] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.721312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.786917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d0d7c-44a4-4f61-9f5b-f2c50ea7b5aa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.812614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.820209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df5a7-d68d-4adb-8927-047cf4b1f776 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.828992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.828992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.828992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.828992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.829323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.829323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.829702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 100/198] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.830494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.835469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:48.855175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.857729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.857729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.857729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.857729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.858427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.927142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e957c6-575a-43fd-b927-356867995bd8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:48.963368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 99/199] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:48 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:48.963368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:48.967858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:48.968839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:48.969363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:48.969780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:48.971361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:48.972855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:48.974992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 101/200] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:48.975602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.046021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.077771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ce60d-370a-490f-b412-b514eaa07b73 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.098137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45a20-4cf2-4e00-b6f1-5c0d15733ef9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.116487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.117436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.117716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.117990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] PATH_INFO: `/v3/users/10d2119ed8ff421196c83d04b8633795` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.119214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] RBAC: Authorizing `identity:get_user(user_id=10d2119ed8ff421196c83d04b8633795)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.120752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.122418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 100/201] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/users/10d2119ed8ff421196c83d04b8633795 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:49.123166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.142974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.144059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.144059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.144230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.176420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.224838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.226513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.257803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb98ef-2384-43b6-8383-974e0bf69b4e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.305943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.306592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.306873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.307153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.307920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=f82eddcc8398428a935a0548a2f9ba32) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:49.308230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.309678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.310014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.315594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 101/202] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:49 2022] GET /identity/v3/role_assignments?user.id=f82eddcc8398428a935a0548a2f9ba32&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:49.316399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.362671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 102/203] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:49.370464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.454186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0ec05-5ad5-405c-a91d-aa9ecd0c45fe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.456029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.504408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f0ff8-f4b9-43a4-9911-8f7ec89e2018 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.514873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.515507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.515834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.516113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.552500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.553158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.553439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.553712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.555120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.557790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.562495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 103/204] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:49 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:49.569036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.610006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.637152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.637152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.732036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d49683-1d15-4c00-9f7f-fd3ae036948f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:49.768981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:49.769599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:49.769877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:49.770180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:49.770923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=f82eddcc8398428a935a0548a2f9ba32) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:49.771345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:49.772807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:49.780220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 104/205] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:49 2022] GET /identity/v3/role_assignments?user.id=f82eddcc8398428a935a0548a2f9ba32&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=28d83235f01b49b98d223883c8317b87 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:49.861455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 102/206] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:49.867273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:49.984747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:49.996182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cf161-9269-4dcf-afa3-8e4d61094b5b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:50.033701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:50.034627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:50.035052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:50.035492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:50.036828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:50.038508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c1b5-0c5b-4aa0-ab31-a4dfdda80b4f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:50.040634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 105/207] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:49 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:50.048741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:50.049206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:50.210347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b39c2-af7d-4a9e-960d-e766646b5e32 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:50.259728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:50.260898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:50.261046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:50.261323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:50.262064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b, user.id=10d2119ed8ff421196c83d04b8633795) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:50.262385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:50.263841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b3301-8faf-4ab6-9762-583d83321164 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:50.269876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 103/208] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:50 2022] GET /identity/v3/role_assignments?user.id=10d2119ed8ff421196c83d04b8633795&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:51.525564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc59f-283b-429e-810f-15f2238e6186 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:51.526143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc59f-283b-429e-810f-15f2238e6186 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:51.526219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc59f-283b-429e-810f-15f2238e6186 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:51.527687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 106/209] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:51.533134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:51.533953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:51.534444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:51.541216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77ef6-7bbe-4f1e-a47f-932248e1a8e9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:51.542024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77ef6-7bbe-4f1e-a47f-932248e1a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:51.542323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77ef6-7bbe-4f1e-a47f-932248e1a8e9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:51.543770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 107/210] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:51.549097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d976c1-b6c6-418c-bbe3-aa2c3add0a8b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:51.549624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d976c1-b6c6-418c-bbe3-aa2c3add0a8b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:51.550132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d976c1-b6c6-418c-bbe3-aa2c3add0a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:51.556949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:51.558768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:51.570894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-14d976c1-b6c6-418c-bbe3-aa2c3add0a8b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:51.572335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-14d976c1-b6c6-418c-bbe3-aa2c3add0a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:51.634278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:51.643849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 108/212] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:51.650314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 104/212] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:51.915492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:51.917761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3408-c1c6-4282-8e12-e3a4532473c9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:51.918078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3408-c1c6-4282-8e12-e3a4532473c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:51.918363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3408-c1c6-4282-8e12-e3a4532473c9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:51.920090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 109/213] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:51.921646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef3408-c1c6-4282-8e12-e3a4532473c9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.019266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.023597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04463e-25e1-4f0a-bdd9-6b2d2674f7dd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.034485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef3408-c1c6-4282-8e12-e3a4532473c9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.059870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.060883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.061318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.061590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.071231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.071891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.072165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.072433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.073975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.074199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:52.075459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bd8fa-7374-4719-968a-ce294261767f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.077251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 110/214] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:52.079194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:52.080311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:52.081299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:52.081929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de13f-3007-4814-9fea-3388c944410a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.081929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de13f-3007-4814-9fea-3388c944410a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.082097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479de13f-3007-4814-9fea-3388c944410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.088175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 105/215] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:52.089866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.102737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-479de13f-3007-4814-9fea-3388c944410a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:52.103980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-479de13f-3007-4814-9fea-3388c944410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:52.149168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.175438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 111/216] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:52.175438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479de13f-3007-4814-9fea-3388c944410a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.200422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff487aa-8aa5-4751-9f08-ded31c6ba204 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.234779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.235384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.235665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.235939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.236603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.238025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050729a-a12f-42b5-b45d-674348e571ff None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.261660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.276656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479de13f-3007-4814-9fea-3388c944410a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.308757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.309361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.309641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.309913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.310654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:52.310957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.312317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ca1ae-18a8-493f-804d-4b8bbe3dde61 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.320135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 112/217] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:52.321260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48ca0-cc27-4ad4-963f-949968c7e0de None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.321545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48ca0-cc27-4ad4-963f-949968c7e0de None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.321819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48ca0-cc27-4ad4-963f-949968c7e0de None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.326317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 113/218] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:52.327038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e48ca0-cc27-4ad4-963f-949968c7e0de None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.346080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 106/219] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:52 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:52.346080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.346080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.346080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.361518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:52.364092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:52.458165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 107/220] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:52.459121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.476217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e48ca0-cc27-4ad4-963f-949968c7e0de None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.519147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.520228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.520681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.521123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] PATH_INFO: `/v3/users/18bf9fd414d74899896b1b9db8df2380` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.521822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.539127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9fd414d74899896b1b9db8df2380)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.541059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.544961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 114/221] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/users/18bf9fd414d74899896b1b9db8df2380 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:52.549923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.563491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.600056 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896e0f6-3685-4ea6-ac02-d83b1b2b1f1c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.653041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.653271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.653939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.653939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.667164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:52.671388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:52.672263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:52.673060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 108/222] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:52.678379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.689191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83787e5-5cc0-41ff-8878-61d7324b71f9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.734846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.735833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.737467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.737953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.809986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e6d63-9532-4b9a-964b-6ac5901ca99a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:52.812620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.820904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.823512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.833402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:52.855894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:52.855894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:52.855894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:52.855894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:52.856306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:52.856685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:52.858866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:52.869150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 109/223] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:52.870019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.971418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 115/224] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:24:52.972272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:52.990257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5133b3-c4b2-4dda-b447-0b6dfce483ee None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.025321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.026019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.026499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.026776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.037632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:53.043633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:53.044515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:24:53.045317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 110/225] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:53.045971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.075383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.113490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542851-784c-4c40-ab04-e7ab01e2b0f4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.123418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.153710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.155567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.155845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.156116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.158772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.160304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.162102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 116/226] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:52 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.162712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.172472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1b83-67e3-4446-986c-b9908b71041b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.214918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.215550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.215829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.216101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.217244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.219370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.226205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 111/227] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.227263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.286803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f19-d77b-4450-942f-3cabfa9afa5e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.323740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.324378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.324656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.324980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.325594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:53.325896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.327302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.341628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 117/228] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.341628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.342474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.353335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a10acd-5bc5-43d3-9c37-bd07978f4e67 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.374815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.399196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.400156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.401163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.401598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.403608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.404920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.407189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 112/229] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.408159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.449277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb60961-8f4f-41c4-8f2c-6ec124146c86 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.485903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.486550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.486830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.487105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.487842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=18bf9fd414d74899896b1b9db8df2380) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:53.488228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.489589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.495671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 118/230] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/role_assignments?user.id=18bf9fd414d74899896b1b9db8df2380&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=d3096edb08e84cd098dda68154bffd4b => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.496478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.567769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0162f0-06ac-47fb-a11e-0d9f37e0e087 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.613900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.614576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.614857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.615129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.618597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.618597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.620317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 113/231] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.625892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.662493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.689822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.693944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003286-ecf4-49b5-ad27-455f23a73078 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.732947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.733618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.733898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.734247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/f82eddcc8398428a935a0548a2f9ba32/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.737538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, user_id=f82eddcc8398428a935a0548a2f9ba32, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.739742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd0ee3-e95c-4f46-99d4-ac9814999a64 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.777386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fecc-41a8-4457-aceb-e7f82d05e259 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:53.812453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:53.813091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:53.813364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:53.813635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:53.814795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:53.816369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:53.818009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 114/232] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:53 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:53.828349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:53.830449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 119/233] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:53 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/f82eddcc8398428a935a0548a2f9ba32/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:53.967147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:53.973159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4aaca-ef1a-468e-8d19-cdca16209c74 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:54.019017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:54.019692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:54.019969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:54.020239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/f82eddcc8398428a935a0548a2f9ba32/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:54.027648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, user_id=f82eddcc8398428a935a0548a2f9ba32, role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:54.029128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead78c0-dcad-4c60-801b-76c560c64f65 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:54.143561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 115/234] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:53 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/f82eddcc8398428a935a0548a2f9ba32/roles/28d83235f01b49b98d223883c8317b87 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:55.010558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948990cd-c5fe-44f6-8f9d-6f3727b517de None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.011609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948990cd-c5fe-44f6-8f9d-6f3727b517de None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.012092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948990cd-c5fe-44f6-8f9d-6f3727b517de None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.014171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 120/235] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:55.028166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb330-e566-4c6a-a169-4773f20e2e5e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.029141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb330-e566-4c6a-a169-4773f20e2e5e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.029656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb330-e566-4c6a-a169-4773f20e2e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.055071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-98efb330-e566-4c6a-a169-4773f20e2e5e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:55.057221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98efb330-e566-4c6a-a169-4773f20e2e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:55.137590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 116/236] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:55.175890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3c58-b58a-40cd-a5d3-044f6193e50f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.176505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3c58-b58a-40cd-a5d3-044f6193e50f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.176975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d3c58-b58a-40cd-a5d3-044f6193e50f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.178547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 121/237] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:55.184801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.185302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.185770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.208211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:55.209730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:55.285354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 117/238] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:55.608990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d3c58-b58a-40cd-a5d3-044f6193e50f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:55.667444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:55.692321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:55.764813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d3c58-b58a-40cd-a5d3-044f6193e50f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:55.827326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:55.827971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.828251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.828522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.847471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbb7d0-28b9-421b-b9e9-899c1049d965 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:55.856731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:55.862254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:55.863117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:24:55.863917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:55.869987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 122/239] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:55.877655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:55.913138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:55.920585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:55.921231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:55.921768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:55.921874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:55.932676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:55.932676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:55.932676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 118/240] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:24:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:55.932676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:56.061366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc86fd-2c83-4ee4-8d35-cc62c99ce72e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:56.083388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:56.098198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4945e80-2c86-4e66-b0d1-9b3ef90cfc50 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:56.110734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:56.111472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.112003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.112550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.113487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:56.113989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:56.115581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbaf3-b453-4407-8710-a6fcb32942ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:56.124726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 123/241] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:56.126528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766815c-233c-4794-8f37-9e074ac92e17 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.127230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766815c-233c-4794-8f37-9e074ac92e17 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.127916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766815c-233c-4794-8f37-9e074ac92e17 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.131262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 124/242] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:56.137470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766815c-233c-4794-8f37-9e074ac92e17 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:56.141047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:56.141666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.141941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.142242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.142923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:56.144933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d70d6-cb87-4c9d-a165-bd1c8c468ec1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:56.228645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:56.247941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766815c-233c-4794-8f37-9e074ac92e17 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:56.262145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 119/243] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:24:55 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:24:56.262145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338cbe-0381-48f1-83cd-6732ad92fe3a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.262145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338cbe-0381-48f1-83cd-6732ad92fe3a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.262145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338cbe-0381-48f1-83cd-6732ad92fe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.299547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5e338cbe-0381-48f1-83cd-6732ad92fe3a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:56.302344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5e338cbe-0381-48f1-83cd-6732ad92fe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:56.320118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:56.320828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.321109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.321379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] PATH_INFO: `/v3/users/22bd6066b4d846fdbd7eaaf83334868e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.351077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] RBAC: Authorizing `identity:get_user(user_id=22bd6066b4d846fdbd7eaaf83334868e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:56.352767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:56.354572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 125/244] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:56 2022] GET /identity/v3/users/22bd6066b4d846fdbd7eaaf83334868e => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:56.360263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:56.461860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 120/245] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:56.534466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723ff6-8f5f-47f8-aaa6-917a894b4817 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:56.535611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173723-3294-4131-8634-d05392a731be None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.535900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173723-3294-4131-8634-d05392a731be None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.536173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173723-3294-4131-8634-d05392a731be None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.537525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 121/246] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:56.542743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfc76-f0f5-4b8c-9d7e-1b6e236fb059 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.543026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfc76-f0f5-4b8c-9d7e-1b6e236fb059 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.543301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfc76-f0f5-4b8c-9d7e-1b6e236fb059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.574939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7fbcfc76-f0f5-4b8c-9d7e-1b6e236fb059 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:56.576138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7fbcfc76-f0f5-4b8c-9d7e-1b6e236fb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:56.577900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:56.578721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.580053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.580325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.595424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:56.668429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:56.675697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 122/247] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:56.688900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:56.690815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:56.798304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a697e2-d5f1-4337-a306-33f3555734bc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.798626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a697e2-d5f1-4337-a306-33f3555734bc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.799147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a697e2-d5f1-4337-a306-33f3555734bc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.801454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 123/248] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:56.805614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 126/249] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:56.808856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.809135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.809407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.811778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:56.837722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:56.839501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:56.870233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:56.925911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 124/250] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:24:56.926736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:56.943569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c6bdb-15ff-4d65-ac97-64682b49d51a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:56.983613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:56.984372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:56.984647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:56.985022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:56.996268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:57.001490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:57.002536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:57.003488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 127/251] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:24:57.004278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.044402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910bb2a-5463-4651-a642-8222fe8cf11c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.075696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.083189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.083819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.084359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.084359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.089048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.089048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.093810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 125/252] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:56 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.094727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.116312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.120781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf397f1-f97c-445f-b5e7-af1b06381d85 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.158689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.159810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.160453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.160919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.172506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:57.178047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:57.178928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:24:57.179738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:57.182697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 128/253] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:57.183764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.206568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9d008-d0a8-401c-9f0a-4b12a38af2c7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.243810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.244937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.245215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.245484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.246181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:57.246493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.248044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.257100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 126/254] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.262086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.310208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb92e-5a31-48ec-b89a-ae5d7b4157da None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.335084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.350371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.350987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.351257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.351528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.352325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=22bd6066b4d846fdbd7eaaf83334868e) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:57.352637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.354686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.366084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 129/255] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/role_assignments?user.id=22bd6066b4d846fdbd7eaaf83334868e&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.366084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.401182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b33d5-98a1-4386-91b4-60aba59fcc80 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.441870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.442077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.442449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.442726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.443890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.447639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:57.447639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.447639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.466043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 127/256] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.468024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.564779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62987-3800-48c1-ba10-4bd376110aa6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.615287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.616812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.617107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.617382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.631177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:24:57.638095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:24:57.638977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:24:57.639754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 130/257] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:24:57.640409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.672765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da2f85-5a9b-4953-8c2e-3651bb9dc4c2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.721221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.740446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.742301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.744444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.744923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] PATH_INFO: `/v3/users/18bf9fd414d74899896b1b9db8df2380` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.746514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9fd414d74899896b1b9db8df2380)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.752460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.754385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 128/258] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/users/18bf9fd414d74899896b1b9db8df2380 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.756565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.815901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:57.848268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedaeec-49d5-4871-b606-1338033153d1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:57.914331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:57.914963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:57.915238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:57.915507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:57.917926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:57.919397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:57.924370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 131/259] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:57.925164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:57.969051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062a6b-f975-4401-867d-9653f498b740 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.054854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.055400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.055680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.055956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.057182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:58.057486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.058901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.062989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735eae9-1bf6-42d8-983e-c7fbb8ba6354 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.088488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 129/260] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:58.089344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:58.098205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.098838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.099116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.099387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.100575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.102145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.104125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 132/261] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:57 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:58.104699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:58.130454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:58.195713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:58.248950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a396a1-2218-434d-b22b-d3e7a4329ce0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.289454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e063a-75e4-4e9b-ab13-7d9358b10b2a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.314682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.315238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.315513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.315780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] PATH_INFO: `/v3/users/f82eddcc8398428a935a0548a2f9ba32` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.317787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] RBAC: Authorizing `identity:get_user(user_id=f82eddcc8398428a935a0548a2f9ba32)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.319269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.322506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 133/262] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:24:58 2022] GET /identity/v3/users/f82eddcc8398428a935a0548a2f9ba32 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:58.324591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:58.357262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.357877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.358180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.358454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.458459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:58.462723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7b252-7d44-4787-a694-edaef8bb47cc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.499921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.500511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.500791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.501066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/18bf9fd414d74899896b1b9db8df2380/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.503059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, user_id=18bf9fd414d74899896b1b9db8df2380, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.504486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.532573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.534116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.580799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 134/263] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:24:58 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/18bf9fd414d74899896b1b9db8df2380/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:24:58.581554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:58.631674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:58.716370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 130/264] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:58.723090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:58.736723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c5170-3d78-4098-8a20-ce06692ea0c8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.776532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.777182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.777437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.777710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.814426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:58.904346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.905839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.944984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2350fe2-75ad-4ae4-9a82-bd54831ef0c9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:58.984568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:58.985205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:58.985482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:58.985753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:58.986931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:58.991024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:58.993690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 131/265] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:58 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:58.996881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:59.036581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:59.110405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 135/266] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:24:59.116032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:59.183550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8563-817b-426e-852c-da2884695801 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:59.218806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:59.219422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.219696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.219966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.220691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=f82eddcc8398428a935a0548a2f9ba32) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:59.220997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:59.222463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900268a9-2c71-455e-8399-2ad25235dacd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:59.229005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 132/267] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:58 2022] GET /identity/v3/role_assignments?user.id=f82eddcc8398428a935a0548a2f9ba32&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:59.253206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:59.335269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9315-662a-49d1-b804-286d7f0992aa None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.335615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9315-662a-49d1-b804-286d7f0992aa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.335894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e9315-662a-49d1-b804-286d7f0992aa None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.336750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e2fb0-67f7-4126-a0ba-95c777d998e2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:59.337238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 133/268] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:24:59.359578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872660b-88f7-4572-adab-639103a94110 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.359905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872660b-88f7-4572-adab-639103a94110 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.362806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872660b-88f7-4572-adab-639103a94110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.391819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:59.392475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.392754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.393026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.395250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:59.396740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:59.398826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 136/269] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:24:59 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:24:59.405466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:24:59.415695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1872660b-88f7-4572-adab-639103a94110 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:59.416987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1872660b-88f7-4572-adab-639103a94110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:59.534410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 134/270] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:59.587221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f2cae-39a4-41bc-9610-a75377815aca None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:24:59.611198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:59.635725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417e49-63b6-451b-b65a-c1c967b53170 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.636117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417e49-63b6-451b-b65a-c1c967b53170 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.636500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417e49-63b6-451b-b65a-c1c967b53170 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.637407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:24:59.638082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.638337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.638629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.639361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=f82eddcc8398428a935a0548a2f9ba32) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:24:59.639674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:24:59.641116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:24:59.642184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 135/271] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:24:59.647544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:24:59.647774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:24:59.648076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:24:59.651399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 137/272] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:24:59 2022] GET /identity/v3/role_assignments?user.id=f82eddcc8398428a935a0548a2f9ba32&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=28d83235f01b49b98d223883c8317b87 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:24:59.673306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:24:59.674625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:24:59.691578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:24:59.774066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 136/273] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:24:59.955961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.071429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce1209-1354-4f61-8fa3-23e4e3e066dd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.110691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.110691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.111425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.111918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.123741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:00.129559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:00.130452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:00.131225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:00.133423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 138/274] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:24:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:00.143121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.144924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.192209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:00.268599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7531ca-9997-48c2-b2f8-f58dafc2ce7b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.270802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2bcea-acbf-4d7b-ad3e-dd93401a7d4b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.307389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.308025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.308299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.308566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.309222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:00.309521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:00.310896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.311009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b5313-aa52-406b-8326-d70e63d1c52d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:00.311572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.311841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.312142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.313376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:00.314940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d2bd8-e637-4b66-bf10-fe3d5e6db268 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:00.316633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 139/275] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:00.317672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86ec1-61a1-4c8b-be32-a893b9c5bafe None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.317954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86ec1-61a1-4c8b-be32-a893b9c5bafe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.318251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86ec1-61a1-4c8b-be32-a893b9c5bafe None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.319606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 140/276] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:00.326171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86ec1-61a1-4c8b-be32-a893b9c5bafe None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.332934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 137/277] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:00.332934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.332934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.332934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.348062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:00.349662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:00.350957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:00.424848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 138/278] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:00.425531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.448753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86ec1-61a1-4c8b-be32-a893b9c5bafe None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.460722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:00.490624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.491928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.492544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.493024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.494555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:00.496247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:00.529722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e8b63-e85b-41b3-895b-22985490b193 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.563610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.564232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.564499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.564761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] PATH_INFO: `/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.568171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:00.579310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] RBAC: Authorizing `identity:get_user(user_id=b95a7c5cc08d41abb8a0a74f02c9b95e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:00.580750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:00.582553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 139/279] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:00.588103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.601636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 141/280] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:00 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:00.704185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.716569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3200e-b84b-456d-9671-666ca5cd8d39 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.762129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.762757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.763034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.763309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.764042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, user.id=b95a7c5cc08d41abb8a0a74f02c9b95e, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:00.764348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:00.765813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:00.771779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 140/281] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/role_assignments?user.id=b95a7c5cc08d41abb8a0a74f02c9b95e&scope.system=all&role.id=d3096edb08e84cd098dda68154bffd4b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:00.861661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36883c-a5a4-4746-b6fa-a08dbbfcc54c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:00.900944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:00.901597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:00.901876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:00.902172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:00.914955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:00.921248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:00.922246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:00.923028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 142/282] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:00.928680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:00.996374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:01.043494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c8de-1c6d-4b9a-90fb-94500f2b9ac8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:01.079566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:01.080225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.080513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.080786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.081455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:01.081755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:01.083177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:01.092640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 141/283] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:01.098306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:01.131304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:01.219283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50da50-ae7c-4e85-a23e-8d89a034cde6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:01.265093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:01.265730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.266013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.266314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] PATH_INFO: `/v3/users/22bd6066b4d846fdbd7eaaf83334868e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.267485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] RBAC: Authorizing `identity:get_user(user_id=22bd6066b4d846fdbd7eaaf83334868e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:01.268987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0339f-19ad-4044-be65-8e400961945b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:01.270677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 143/284] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:01 2022] GET /identity/v3/users/22bd6066b4d846fdbd7eaaf83334868e => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:01.276519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:01.307208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868183d-349e-4fd4-8a87-35f4d4cbc9ab None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.307708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868183d-349e-4fd4-8a87-35f4d4cbc9ab None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.307708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868183d-349e-4fd4-8a87-35f4d4cbc9ab None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.309225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 144/285] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:01.314341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.314621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.314894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.347818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:01.349120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:01.395011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ff98d-2771-4bf9-b77b-bbe45c3dd8f2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:01.419827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:01.434270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:01.434913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.435393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.435498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.436578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:01.438074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:01.440008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 142/286] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:01 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:01.443591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 145/287] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:01.451274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:01.567293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd7ae-0dea-4e2e-8fc7-78abbcf34427 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:01.613327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:01.613996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.614335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.614670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/22bd6066b4d846fdbd7eaaf83334868e/roles/2aa07c7012a14708a05efb12b4f81849` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.616689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, user_id=22bd6066b4d846fdbd7eaaf83334868e, role_id=2aa07c7012a14708a05efb12b4f81849)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:01.618310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:01.730889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 143/288] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:01 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/users/22bd6066b4d846fdbd7eaaf83334868e/roles/2aa07c7012a14708a05efb12b4f81849 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:01.750202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:01.777864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:01.862684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83aa7-27ef-4df9-b350-dbac429c507d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:01.901964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:01.903130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:01.903602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:01.904080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:01.916871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:01.922560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:01.923505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:01.924292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:01.927086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 146/289] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:01.933872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:02.000469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:02.069847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e6d54-29eb-4969-a350-268d5843c7a7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:02.114332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:02.115356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:02.115826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:02.116298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:02.117161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:02.117649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:02.119464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:02.134405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 144/290] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:02.140305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:02.226168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:02.245278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbec909-75bd-496d-a971-ce0d8dbfa21a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:02.280326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:02.281286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:02.281755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:02.282249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] PATH_INFO: `/v3/users/18bf9fd414d74899896b1b9db8df2380` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:02.283583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] RBAC: Authorizing `identity:get_user(user_id=18bf9fd414d74899896b1b9db8df2380)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:02.285223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:02.287298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 147/291] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:02 2022] GET /identity/v3/users/18bf9fd414d74899896b1b9db8df2380 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:02.293408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:02.429662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24872542-bddd-4750-a7ea-583b6e1d3773 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:02.445616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:02.472032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:02.472636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:02.472909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:02.473175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:02.571114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:02.573320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:02.705570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 145/292] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:02.711024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:02.813220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43fc5e-69b2-4cf6-8a3c-266c611ef156 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:02.847896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:02.848446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:02.848720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:02.848986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:02.850130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:02.851556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee108-880c-49e3-b902-6659eb68a4a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:02.854003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 148/293] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:02 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:02.859796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:02.878477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:02.987938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baf8a-623e-4f0b-bc43-c01ca6a56ce0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:03.022620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:03.023225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.023502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.023767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.024576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=18bf9fd414d74899896b1b9db8df2380) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:03.024882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:03.026336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18405a-524b-4b98-958d-782fc16d9a50 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:03.032419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 146/294] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:02 2022] GET /identity/v3/role_assignments?user.id=18bf9fd414d74899896b1b9db8df2380&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=d3096edb08e84cd098dda68154bffd4b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:03.355360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c515d55-f9d4-4510-96a2-a9bddb9e9601 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.355739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c515d55-f9d4-4510-96a2-a9bddb9e9601 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.355913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c515d55-f9d4-4510-96a2-a9bddb9e9601 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.357590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 149/295] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:03.364069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1c1-5fe0-4aba-8db0-52144b54a1b6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.364567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1c1-5fe0-4aba-8db0-52144b54a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.364641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1c1-5fe0-4aba-8db0-52144b54a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.392256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4c8ca1c1-5fe0-4aba-8db0-52144b54a1b6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:03.393468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4c8ca1c1-5fe0-4aba-8db0-52144b54a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:03.468875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 147/296] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:03.484393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec006-da3f-4e89-a4c6-68da55e3869a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.484796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec006-da3f-4e89-a4c6-68da55e3869a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.485070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec006-da3f-4e89-a4c6-68da55e3869a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.486469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 150/297] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:03.495035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338b79-00a6-46a1-a912-29bd80e4971a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.495035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338b79-00a6-46a1-a912-29bd80e4971a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.495035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338b79-00a6-46a1-a912-29bd80e4971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.502279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:03.514219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d6338b79-00a6-46a1-a912-29bd80e4971a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:03.515423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d6338b79-00a6-46a1-a912-29bd80e4971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:03.593729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 148/298] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:03.761436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec006-da3f-4e89-a4c6-68da55e3869a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:03.781423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:03.819004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c78f-1cf9-4b80-b934-21705df4bdfb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.819287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c78f-1cf9-4b80-b934-21705df4bdfb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.819562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c78f-1cf9-4b80-b934-21705df4bdfb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.820921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 149/299] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:03.827103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.827602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.828187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.847375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:03.848868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:03.871132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec006-da3f-4e89-a4c6-68da55e3869a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:03.906724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:03.907347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:03.907627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:03.907895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:03.916231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 150/300] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:03.916982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:03.925171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:03.930940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:03.931785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:03.932622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:03.935439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 151/301] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:03.942091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.005000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.046794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5c284-900f-4cb4-b990-d3e0a9c76423 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.047680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.076399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9f843-8482-4467-9e90-7950d6f0d311 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.083605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.084469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.084848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.085218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.086468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.088098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.090012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 151/302] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:04.095433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.117745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.119388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.119814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.120225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.121060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:04.121513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.123479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.134757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 152/303] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:04.146108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.223176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb07e0c-31bc-4c3d-a582-a33860c02681 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.244757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16829b-8534-408e-b4cd-27f8dffab4d7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.252446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.257207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.257819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.258140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.258397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] PATH_INFO: `/v3/groups` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.259556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.261005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.280719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.281341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.281613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.281880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] PATH_INFO: `/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.283016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] RBAC: Authorizing `identity:get_user(user_id=b95a7c5cc08d41abb8a0a74f02c9b95e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.284501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.289142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 153/304] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:04 2022] GET /identity/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:04.289142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.301252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.342664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 152/305] 10.222.0.60 () {60 vars in 1134 bytes} [Fri Sep 9 11:25:04 2022] POST /identity/v3/groups => generated 229 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:25:04.343794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.411510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97500df-0ca0-4761-9230-1ccd9b26ef00 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.452449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.453687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.454231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.455580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.470316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:04.476716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:04.477738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:04.478582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:04.484526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 154/306] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:04.485328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.501099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865deae-9159-4602-a452-d11345af85f3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.556152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.557158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.557646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.557983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] PATH_INFO: `/v3/system/users/b95a7c5cc08d41abb8a0a74f02c9b95e/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.559605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b95a7c5cc08d41abb8a0a74f02c9b95e, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.561002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715df2d-fdd2-43c7-b303-493289b10047 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.579200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.612485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c121-20cf-4bc0-b895-ee3d78d39a81 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.646602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.647095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.647373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.647706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.648375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:04.648676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.651721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 153/307] 10.222.0.60 () {58 vars in 1334 bytes} [Fri Sep 9 11:25:04 2022] PUT /identity/v3/system/users/b95a7c5cc08d41abb8a0a74f02c9b95e/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:04.652835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f716-b533-4009-9f6e-c5a67f9dd123 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.653120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f716-b533-4009-9f6e-c5a67f9dd123 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.653397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915f716-b533-4009-9f6e-c5a67f9dd123 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.656649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 154/308] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:04.664119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.664262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.664502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.664985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.667039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 155/309] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:04.672795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.694869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:04.698490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:04.779256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.799451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a9185-5d02-4606-a7ad-3da34175b18c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:04.806106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 155/310] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:04.839049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:04.840687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:04.841514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:04.842103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:04.842589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] PATH_INFO: `/v3/users/88063b00fa10432ca0f6c75201680624` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:04.857746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] RBAC: Authorizing `identity:get_user(user_id=88063b00fa10432ca0f6c75201680624)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:04.859482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:04.861374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 156/311] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:04 2022] GET /identity/v3/users/88063b00fa10432ca0f6c75201680624 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:04.867531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:04.972341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eef485-f803-4290-bb3a-d218e1247c42 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:05.007466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:05.008068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:05.008392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:05.009182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:05.009930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, user.id=88063b00fa10432ca0f6c75201680624, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:05.010260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:05.011691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:05.017253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 156/312] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Sep 9 11:25:04 2022] GET /identity/v3/role_assignments?user.id=88063b00fa10432ca0f6c75201680624&scope.system=all&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:05.087175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:05.200593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed4e5-97e8-483b-be49-0d7e3d84f641 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:05.236406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:05.237044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:05.237323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:05.237596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:05.248359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:05.253268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:05.254332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:05.255311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:05.258161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 157/313] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:05.261155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:05.312765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:05.365001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d8630-61b1-4371-a7a0-29175011c611 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:05.406696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27624-d850-4441-af17-1ead09620833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:05.407319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27624-d850-4441-af17-1ead09620833 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:05.407593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27624-d850-4441-af17-1ead09620833 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:05.407859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27624-d850-4441-af17-1ead09620833 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:05.408528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c27624-d850-4441-af17-1ead09620833 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:05.408824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c27624-d850-4441-af17-1ead09620833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:05.410227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c27624-d850-4441-af17-1ead09620833 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:05.419367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 157/314] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:05.424951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:05.479032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:05.531375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525fef9-a673-4901-be7c-0f3f9b24221c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:05.566039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:05.566690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:05.566968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:05.567237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] PATH_INFO: `/v3/users/22bd6066b4d846fdbd7eaaf83334868e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:05.568370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] RBAC: Authorizing `identity:get_user(user_id=22bd6066b4d846fdbd7eaaf83334868e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:05.569806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:05.576377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 158/315] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:05 2022] GET /identity/v3/users/22bd6066b4d846fdbd7eaaf83334868e => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:05.577734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27624-d850-4441-af17-1ead09620833 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:05.686044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27624-d850-4441-af17-1ead09620833 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:05.700040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:05.722826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:05.723563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:05.723838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:05.724104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:05.797457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:05.798848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:05.902453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 158/316] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:05.907856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:06.013323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b6d1-d434-48f3-bef4-d378d5aaa5cb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:06.029619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:06.047612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:06.048182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.048459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.048733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.049855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:06.051309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6eb7a-bd7b-4ab4-8cc4-50026b260a22 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:06.058162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 159/317] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:05 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:06.059831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:06.078376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:06.165348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e36709-0e42-4bf4-b7a0-848769b0e6fe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:06.203067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:06.203667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.203938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.204202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.204916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, scope.project.id=e189ed93b20c4352a2f666cc5881218a, user.id=22bd6066b4d846fdbd7eaaf83334868e) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:06.205220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:06.206677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb83d-3b9d-48d7-9a51-2f931af91c8d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:06.213419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 159/318] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:06 2022] GET /identity/v3/role_assignments?user.id=22bd6066b4d846fdbd7eaaf83334868e&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:06.685190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4698016-f55a-4bac-84ca-bf04e7a05985 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.685433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4698016-f55a-4bac-84ca-bf04e7a05985 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.685755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4698016-f55a-4bac-84ca-bf04e7a05985 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.687157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 160/319] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:06.692320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8bb2-d462-4eec-aa17-5969b4636cee None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.696806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8bb2-d462-4eec-aa17-5969b4636cee None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.696806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8bb2-d462-4eec-aa17-5969b4636cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.712514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-200d8bb2-d462-4eec-aa17-5969b4636cee None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:06.713770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-200d8bb2-d462-4eec-aa17-5969b4636cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:06.783574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 160/320] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:06.899635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cc527-ceda-4a85-b200-15fce283df4d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.899978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cc527-ceda-4a85-b200-15fce283df4d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.900182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cc527-ceda-4a85-b200-15fce283df4d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.901644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 161/321] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:06.906652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:06.906941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:06.907214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:06.919398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:06.925860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:06.927077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:06.994820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 161/322] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:07.052879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cc527-ceda-4a85-b200-15fce283df4d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.155852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cc527-ceda-4a85-b200-15fce283df4d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.190728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.191358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.191774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.191976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.193067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:07.194530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e989b-f815-4f92-b1a9-d5a13dfdb3c2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:07.197086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 162/323] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:07.202552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.254682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac83ac-3bc5-4da3-a98e-24d9160457c1 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.254968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac83ac-3bc5-4da3-a98e-24d9160457c1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.255154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac83ac-3bc5-4da3-a98e-24d9160457c1 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.258095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 163/324] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:07.260439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac83ac-3bc5-4da3-a98e-24d9160457c1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.304090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017c18c-7e9f-4600-bd54-cf7b8cede84e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.313547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:07.338959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.339563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.339839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.340106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] PATH_INFO: `/v3/groups` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.340736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:07.342201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:07.346876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:07.361784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac83ac-3bc5-4da3-a98e-24d9160457c1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.395075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.395743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.395969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.397795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.407112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-58301ef8-c0ba-4265-943a-f90051d0281a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:07.411911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:07.412911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:07.413728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 164/325] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:07.415446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.415731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.416004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.434948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 162/326] 10.222.0.60 () {60 vars in 1134 bytes} [Fri Sep 9 11:25:07 2022] POST /identity/v3/groups => generated 226 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:07.434948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.435225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:07.436269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:07.515932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 165/327] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:07.535882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5d878-1e16-40ef-8ff3-c07939aef8a0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.574308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.574925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.575203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.575471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.576145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:07.576445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:07.577837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:07.587958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 163/328] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:07.593541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.639446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:07.697710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e080e-4985-4c43-b12a-8a9b26f66485 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.732756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.733692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.734202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.734646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] PATH_INFO: `/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.735948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] RBAC: Authorizing `identity:get_user(user_id=b95a7c5cc08d41abb8a0a74f02c9b95e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:07.737561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:07.739366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 166/329] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/users/b95a7c5cc08d41abb8a0a74f02c9b95e => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:07.745929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.783344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:07.828521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:07.881318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ee1a6-e4b9-45cf-85e6-4fe06a1ac621 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.907986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e570fba-0681-41e0-84cc-6d5f21a18259 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:07.924319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:07.933062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.933954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.934519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.934988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.935934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] RBAC: Adding query filter params (role.id=d3096edb08e84cd098dda68154bffd4b, user.id=b95a7c5cc08d41abb8a0a74f02c9b95e, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:07.936249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:07.937711 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:07.953779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 164/330] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/role_assignments?user.id=b95a7c5cc08d41abb8a0a74f02c9b95e&scope.system=all&role.id=d3096edb08e84cd098dda68154bffd4b => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:07.956657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:07.957322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:07.957602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:07.957894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:07.972348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:07.978933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:07.979886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:07.980668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 167/331] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:07.987904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:08.104006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd307840-5806-4b94-868b-bbacb6d6be25 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:08.122517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:08.142890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:08.143530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:08.143872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:08.144146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:08.144880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:08.145184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:08.146999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:08.157288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 165/332] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:08.162888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:08.276002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960637-3fe9-40c8-b019-0a669e51be5b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:08.313971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:08.314642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:08.314920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:08.315193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] PATH_INFO: `/v3/users/88063b00fa10432ca0f6c75201680624` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:08.316379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] RBAC: Authorizing `identity:get_user(user_id=88063b00fa10432ca0f6c75201680624)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:08.317836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1620-7188-400a-adfe-87fbb03d6dee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:08.319511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 168/333] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:08 2022] GET /identity/v3/users/88063b00fa10432ca0f6c75201680624 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:08.326099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:08.465394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cea950-dc9d-4805-9a49-855afc649f56 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:08.509934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:08.510629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:08.510906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:08.511183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] PATH_INFO: `/v3/system/users/88063b00fa10432ca0f6c75201680624/roles/2aa07c7012a14708a05efb12b4f81849` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:08.515512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=88063b00fa10432ca0f6c75201680624, role_id=2aa07c7012a14708a05efb12b4f81849)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:08.516972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea3e8-b282-472c-abc1-3ca190bd2a77 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:08.587527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 166/334] 10.222.0.60 () {58 vars in 1334 bytes} [Fri Sep 9 11:25:08 2022] PUT /identity/v3/system/users/88063b00fa10432ca0f6c75201680624/roles/2aa07c7012a14708a05efb12b4f81849 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:10.394777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930fca-1ae8-4ee7-aa65-a6a98d834483 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.395299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930fca-1ae8-4ee7-aa65-a6a98d834483 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.395299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930fca-1ae8-4ee7-aa65-a6a98d834483 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.396942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 169/335] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:10.402441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.402876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.403025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.427965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:10.429245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:10.468841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4d2af-7470-41ec-a8ef-72810b806d8f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.469175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4d2af-7470-41ec-a8ef-72810b806d8f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.469391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4d2af-7470-41ec-a8ef-72810b806d8f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.471085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:10.474176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 170/336] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:10.498760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555543-1424-4c95-82aa-37b480af93a0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.499363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555543-1424-4c95-82aa-37b480af93a0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.499852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555543-1424-4c95-82aa-37b480af93a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.511177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 167/337] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:10.531750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0f555543-1424-4c95-82aa-37b480af93a0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:10.534723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f555543-1424-4c95-82aa-37b480af93a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:10.572582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:10.622078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 171/338] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:10.834979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:10.924901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420212bd-fa88-4030-876f-f7047d2e64ad None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.925552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420212bd-fa88-4030-876f-f7047d2e64ad None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.926078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420212bd-fa88-4030-876f-f7047d2e64ad None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.927885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 172/339] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:10.943512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:10.944138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:10.944628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:10.958990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7224a-60d2-4c77-abb3-ef258e32526d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:10.981696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:10.983278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:10.996767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.002160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.002512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.002842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.025679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:11.031828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:11.032774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:11.033608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:11.038698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 168/340] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:11.045454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.082425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.100794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 173/341] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:11.100794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.154886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92009b0-9b0f-471a-8e5a-59bf9611f069 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.194016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.194682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.194978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.195781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.196684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:11.196988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:11.198490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:11.206450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049538b-1114-471b-a5ce-80efa700c997 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.207797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 169/342] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:11.212901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.215022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.326770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.327397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17e1ba-0926-42c8-91c5-223f9f72c815 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.328041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.328539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.329018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.330377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.365240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.366177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:11.366321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.366693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.367068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:11.372360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:11.373368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:11.374197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:11.389469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 174/343] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:11.395518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.448638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:11.450446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:11.558456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.565656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 170/344] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:11.575473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.576633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843104d-96de-4a93-b44b-3e9df657971d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.642650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.643503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.643944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.645231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.646143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:11.646842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:11.654194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:11.670501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 175/345] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:11.678367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.712185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.759173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8345c-bc4e-4c1f-986c-c9266ab13897 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.808983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.809773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.810047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.810674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.812086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:11.814206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:11.816547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 171/346] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:11.817908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:11.819720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f299a-93b5-48a5-bfef-7cab13b8f25a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.860371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.860984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.861263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.861535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.894647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.942678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd87ab-914a-4406-a580-8ead31bbd196 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:11.950835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:11.952725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:11.981342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:11.984377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:11.985034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:11.985312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:11.985585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:11.999473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-093b0a68-2eba-4b92-ad42-3f6f40b62683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:12.006545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:12.007622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 172/347] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:12.008501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86964b7-d3f2-4eb0-b2ab-fae9e8d59484 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.009232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86964b7-d3f2-4eb0-b2ab-fae9e8d59484 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.009232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86964b7-d3f2-4eb0-b2ab-fae9e8d59484 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.010365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 173/348] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:12.011145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86964b7-d3f2-4eb0-b2ab-fae9e8d59484 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.103058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 176/349] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:12.104862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.157439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86964b7-d3f2-4eb0-b2ab-fae9e8d59484 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.186319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.201133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.201582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.201582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.201841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.209599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.212049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d12c18-a3d9-4460-bbf8-ed9b5900220a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.214206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 174/350] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:12.216276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.216834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.217326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.236445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013bab0-fd76-4549-909e-d01cbdeac861 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.241498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:12.243571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:12.263089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.277907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.278693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.278849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.279082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.279773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:12.280072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.281605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.297052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 177/351] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:12.297754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.341455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 175/352] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:12.342365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.432438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33c99a-47d9-4851-ae0a-02e73d329f0e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.466210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d5f0c-fddd-4d07-8b2d-9345bf451ff5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.471034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.471738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.474273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.474625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.476266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.478383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10603f17-8949-49fd-ab1a-edcae71891ec None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.480669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 178/353] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:12.481660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc602-2e7f-4c9a-bdf9-591dc1492d18 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.487095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc602-2e7f-4c9a-bdf9-591dc1492d18 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.487095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fc602-2e7f-4c9a-bdf9-591dc1492d18 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.491885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 179/354] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:12.491885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fc602-2e7f-4c9a-bdf9-591dc1492d18 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.493340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.523359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.524715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.525437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.525890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.527075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:12.527955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.530829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.538141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 176/355] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=28d83235f01b49b98d223883c8317b87 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:12.539897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.552062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.621565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fc602-2e7f-4c9a-bdf9-591dc1492d18 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.677653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.678483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.678762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.679082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.690144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585576-b76d-4970-b436-ab7dbcd410de None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:12.740896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:12.741878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.742856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.743278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.747241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.753738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e1dce-d0d3-4c24-8a37-1610e91bba8f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.753738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 177/356] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:12.753738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:12.753997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:12.754238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:12.779754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:12.784208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:12.784587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:12.785818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:12.849956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.854023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:12.879847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 178/357] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:12.880781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.913203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 180/358] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:12.914474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:12.995179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc64347-34ea-491f-a42f-9bf5c38db15f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.036203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.036852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.037195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.038889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.049770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56d81-2d26-4070-b07f-9a13562c0951 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.056216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-794bc93b-8e72-4b22-9594-6d41a2971bf7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:13.061237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:13.062225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:13.063007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 179/359] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:13.064539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd7f8-b57d-474c-b988-017bf6283348 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.065001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd7f8-b57d-474c-b988-017bf6283348 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.065286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd7f8-b57d-474c-b988-017bf6283348 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.066581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 180/360] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:13.067282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd7f8-b57d-474c-b988-017bf6283348 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.107064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.107682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.107964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.108230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.109036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:13.109346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.110805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.119927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 181/361] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:12 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=d3096edb08e84cd098dda68154bffd4b => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:13.120839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.144291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.162618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.235401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd7f8-b57d-474c-b988-017bf6283348 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.290612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.291680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.291978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.292254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.293735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.296687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699f9db-9b5a-4834-989b-22d43631e188 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.298932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 181/362] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:13.300896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.301344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.301752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.314448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b3096-1e01-4970-b28f-6b8c85a7031d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.326221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:13.328358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:13.360809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.361443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.361722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.362156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.362773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:13.363131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.364532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.375792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 182/363] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:13.379083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.424957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 182/364] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:13.425903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.473720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.481797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.512939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e074fac-8b29-437e-9c9c-a68d437e0242 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.579875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9873a3e-0a81-4c25-aceb-ad95d5ecae49 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.618874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.618874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.618874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.618874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.622596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.623457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.623944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.624404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.625721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.627430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.629761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 183/365] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:13.630902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.670832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:13.677219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:13.678478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:25:13.680113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:13.683345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 183/366] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:13.683345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:13.758489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355bd0-8bb4-4e96-9ef7-57d11acf5651 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.762368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.797215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.800466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.801013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.801489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.891527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.892843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.905047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f7e18-a185-46c0-8f22-5d9695e6e5cf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:13.942389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:13.943358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:13.943831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:13.944298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:13.946098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=e189ed93b20c4352a2f666cc5881218a) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:13.946606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:13.948296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:13.951439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:13.955244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 184/367] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=d3096edb08e84cd098dda68154bffd4b => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:13.958339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.013521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 184/368] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:14.014426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.019023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.088345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3bf7f-f853-4f37-8930-11a7e20e7fe6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.127658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.128248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.128527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.128853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.129536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:14.129839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.131485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.141117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 185/369] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.142007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.145398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c581-47f3-43b3-9b73-aacbfe77ea28 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.204694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.205722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.206229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.206709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.221738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.228278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:14.242080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:14.243141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:14.244036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:14.246433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 185/370] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:14.247458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.262404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a75c26-71de-4b90-896a-a9ef988245bf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.297577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.298339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.298689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.299033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.300362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.302069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.306079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 186/371] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.306079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.363230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.426223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957e818-6e62-4c74-a4dd-463ed44d6073 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.459286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.460204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a8b67-bc90-489a-9e39-013bcabcbce8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.465357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.465972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.466296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.466572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.475758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:14.475758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.475758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.479530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 187/372] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.480438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.531965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.532630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.537633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.537633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] PATH_INFO: `/v3/users/88063b00fa10432ca0f6c75201680624` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.537633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] RBAC: Authorizing `identity:get_user(user_id=88063b00fa10432ca0f6c75201680624)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.537633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.537633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 186/373] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/users/88063b00fa10432ca0f6c75201680624 => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.538343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.651251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95be33-a2e4-4b22-b5af-fe7488e3e99d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.704192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234cda-421e-47f8-ac48-37186d6d5099 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.710501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.711227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.711509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.711782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.712909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.714383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.716106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 188/374] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.716998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:14.768895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.775703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.780445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.780445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.780445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.814427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:14.880471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.881972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.890176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0f674-f9d1-491d-af71-872fbb504d5f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:14.953949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:14.954637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:14.954858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:14.955127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:14.955855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] RBAC: Adding query filter params (role.id=2aa07c7012a14708a05efb12b4f81849, user.id=88063b00fa10432ca0f6c75201680624, scope.system=all) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:14.956161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:14.957601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:14.981360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 189/375] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/role_assignments?user.id=88063b00fa10432ca0f6c75201680624&scope.system=all&role.id=2aa07c7012a14708a05efb12b4f81849 => generated 510 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:14.982281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:15.078726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 187/376] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:15.085046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:15.130465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:15.192468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4e6f0-27d3-4b5e-9cb3-c27e5f5dc248 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:15.229066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:15.236289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde6b9f-19af-4878-a774-b5517c6b22b9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:15.265030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:15.265667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.265945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.266259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.267010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=e189ed93b20c4352a2f666cc5881218a) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:15.267317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:15.268807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:15.288289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 190/377] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:14 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=28d83235f01b49b98d223883c8317b87 => generated 260 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:15.299248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:15.299919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.300192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.300462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] PATH_INFO: `/v3/projects/e4bd719a3a8247ec9289af5d26132989` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.301678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=e4bd719a3a8247ec9289af5d26132989)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:15.303226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8f636-b90b-412c-a315-8d1fb3b5c6e2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:15.305936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 188/378] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:15 2022] GET /identity/v3/projects/e4bd719a3a8247ec9289af5d26132989 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:15.311840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:15.435175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f74165-3454-45c9-b3e7-8ace32fbf969 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:15.479892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:15.480510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.480788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.481062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] PATH_INFO: `/v3/groups/44f2945b1891407aaba87e9037ca43cf` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.484782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24519da-958e-43ce-90df-6211df5704c8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.485057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24519da-958e-43ce-90df-6211df5704c8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.485322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24519da-958e-43ce-90df-6211df5704c8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.488084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 189/379] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:15.493076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] RBAC: Authorizing `identity:get_group(group_id=44f2945b1891407aaba87e9037ca43cf)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:15.494916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8129-52b7-4a3b-9146-bfcaa01c2c23 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:15.497155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 191/380] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:15 2022] GET /identity/v3/groups/44f2945b1891407aaba87e9037ca43cf => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:15.503535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24519da-958e-43ce-90df-6211df5704c8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:15.507265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.507769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.508243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.536232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:15.537786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:15.609313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:15.639411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 192/381] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:15.717291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:15.721502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24519da-958e-43ce-90df-6211df5704c8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:15.762227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:15.762755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:15.763031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:15.763299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:15.764022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] RBAC: Adding query filter params (group.id=44f2945b1891407aaba87e9037ca43cf, role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e4bd719a3a8247ec9289af5d26132989) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:15.764327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:15.765761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:15.773080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 190/382] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:15 2022] GET /identity/v3/role_assignments?group.id=44f2945b1891407aaba87e9037ca43cf&scope.project.id=e4bd719a3a8247ec9289af5d26132989&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:15.942655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:16.074017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b52cf-9deb-47e5-8923-73679f75cb0c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:16.113829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:16.114541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.114819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.115090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.126786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-5301b9bf-d48b-4953-bc6e-cca10d2699e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:16.131726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:16.132639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:16.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 193/383] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:16.139016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:16.176846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20137297-5e17-4dfe-b1f4-75d46dec7d00 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.177703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20137297-5e17-4dfe-b1f4-75d46dec7d00 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.177983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20137297-5e17-4dfe-b1f4-75d46dec7d00 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.180174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 194/384] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:16.187271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20a42-5351-442f-941b-24382f26e647 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.187554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20a42-5351-442f-941b-24382f26e647 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.187997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20a42-5351-442f-941b-24382f26e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.209526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-32b20a42-5351-442f-941b-24382f26e647 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:16.210881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-32b20a42-5351-442f-941b-24382f26e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:16.243659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:16.276654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb336d6-cc58-4525-ad3f-001a8f55b833 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:16.318134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 195/385] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:16.333735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:16.334297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.334606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.334843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.335625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:16.335924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:16.337293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:16.348953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 191/386] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:16.355306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20a42-5351-442f-941b-24382f26e647 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:16.500736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20a42-5351-442f-941b-24382f26e647 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:16.541925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:16.542586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.542860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.543130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.546718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:16.548159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:16.549738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 196/387] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:16 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:16.555467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:16.652810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:16.809647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:16.820819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ef1c-4430-400c-9018-a2a0ee5f2979 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:16.883630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:16.893108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:16.899957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:16.899957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:16.899957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:16.899957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:16.899957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:16.900758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 192/388] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:16 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:16.908698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.029838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5d0d0-2172-448b-b83a-5e738a5da8c7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:17.103108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:17.109486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:17.110464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.110900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.111314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.117030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3bf86-216a-4dce-977d-fdde017000b2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:17.141345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:17.150135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:17.151120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:17.151925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:17.155177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 197/389] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:17.173269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.202937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:17.204017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.205455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.205976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/groups/e09314df6c2946d6be360e1809e86bba/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.211977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, group_id=e09314df6c2946d6be360e1809e86bba, role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:17.213629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b079f-f30d-487c-b735-feae277c0a9e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:17.305646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 193/390] 10.222.0.60 () {58 vars in 1442 bytes} [Fri Sep 9 11:25:16 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/groups/e09314df6c2946d6be360e1809e86bba/roles/28d83235f01b49b98d223883c8317b87 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:17.307361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246aae3-e588-4de9-a2f8-fa905f5ee98c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.307645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246aae3-e588-4de9-a2f8-fa905f5ee98c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.308003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246aae3-e588-4de9-a2f8-fa905f5ee98c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.311965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 194/391] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:17.318933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.319235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.319521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.331215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:17.339547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068edcfd-5e59-4d94-afb9-5965724d6530 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:17.350354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:17.352714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:17.389555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:17.390635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.391049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.391545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.392389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:17.392856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:17.395203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:17.406136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 198/392] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:17.411307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.456508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 195/393] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:17.577004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34250244-347e-4462-ab4e-9a1d6ed3bcd8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:17.615285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:17.616140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.617085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.617489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.618828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:17.620414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:17.622437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 199/394] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:17 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:17.627767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.749979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:17.768491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed928e1e-ed3f-4b32-867a-e20a80307aec None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:17.813264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:17.813998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:17.817003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:17.817284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:17.818601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:17.821007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:17.823127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 196/395] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:17 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:17.828364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.837884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:17.879425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:18.036600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a850-d291-4c24-9307-4879527c13de None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.042537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e814e-b54c-4fb8-be56-61284737b012 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.111610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.112263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.112540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.112812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.113889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.114792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.115072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.115344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/groups/e09314df6c2946d6be360e1809e86bba/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.119190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9e48dbb6ee64fd082bf26f66d88058b, group_id=e09314df6c2946d6be360e1809e86bba, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:18.120634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f171-0df2-4393-aeb1-77d4820fb3e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:18.134306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:18.139187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:18.144902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:18.145706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 197/396] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:18.146948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:18.163546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:18.204029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 200/397] 10.222.0.60 () {58 vars in 1442 bytes} [Fri Sep 9 11:25:17 2022] PUT /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b/groups/e09314df6c2946d6be360e1809e86bba/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:18.220895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce0bda-d4a7-484e-ba19-eb8a7557f0c0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.221184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce0bda-d4a7-484e-ba19-eb8a7557f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.221464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce0bda-d4a7-484e-ba19-eb8a7557f0c0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.222958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 201/398] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:18.245632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.245913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.246216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.277216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:18.281571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:18.329527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f150e4-a880-4c38-b0a1-5b4218f80904 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.379021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.382083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.382083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.382083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.382083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:18.382083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:18.387098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:18.399002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 198/399] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:18.404510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:18.427380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 202/400] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:18.489631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:18.524806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d7bf7-6e67-40ee-a6fa-b640a61eb4f7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.571463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.572089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.572361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.572625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.573737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:18.575199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:18.577058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 199/401] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:18 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:18.582689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:18.599918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:18.710075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15b4fa-5b26-4a2f-9461-14bb131ec469 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.760276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.760903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.761224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.761494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.762631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:18.764123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:18.765805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 203/402] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:18 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:18.771622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:18.833744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:18.893914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc9cb8-34c6-4c25-b701-4ac4e1ce5d8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.920884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:18.931804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.932732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.933174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.939889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/groups/e09314df6c2946d6be360e1809e86bba/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.939889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, group_id=e09314df6c2946d6be360e1809e86bba, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:18.939889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fb68a-8b79-48d3-b8da-07ea5db031a0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:18.940584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b9f1-34ee-4969-af83-43f62880cd52 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:18.974361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:18.975215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:18.979534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:18.979534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:18.987315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-aee9cdcc-e3ec-475d-a1e3-564c82149469 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:18.992550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:18.993458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:18.994297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 204/403] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:19.000151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87220ff-0b32-4817-a9b3-7f489982c065 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.000151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87220ff-0b32-4817-a9b3-7f489982c065 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.000151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87220ff-0b32-4817-a9b3-7f489982c065 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.000151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 205/404] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:19.001530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87220ff-0b32-4817-a9b3-7f489982c065 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.072876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 200/405] 10.222.0.60 () {58 vars in 1442 bytes} [Fri Sep 9 11:25:18 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/groups/e09314df6c2946d6be360e1809e86bba/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:19.075329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.075848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.076597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.086311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:19.104938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:19.107171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:19.117811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87220ff-0b32-4817-a9b3-7f489982c065 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.161559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.162210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.162569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.163110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.163855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:19.164691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:19.165538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:19.177078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 206/406] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:19.185275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.193661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:19.218499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 201/407] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:19.315751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509922-de9c-4fb5-ab4a-836491fce073 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.354977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.355947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.356427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.356936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.358314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:19.360022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:19.365237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 207/408] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:19.368307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.474899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268f67-2c56-4e61-a798-d17d68300b50 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.522637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:19.531594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.532240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.532519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.532794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.534003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:19.538558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:19.538558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 202/409] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:19.543435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.554908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.676825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d88fe-d8d4-4382-819e-f5cc043e199a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.699851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:19.720139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.720890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.721172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.721445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.739337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:19.746361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:19.747293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:19.748100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 203/410] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:19.753679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:19.770083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db415-d721-421a-b409-4aa3d157d8b9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.833594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.834289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.834570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.834905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a/groups/e09314df6c2946d6be360e1809e86bba/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.838262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] RBAC: Authorizing `identity:create_grant(project_id=e189ed93b20c4352a2f666cc5881218a, group_id=e09314df6c2946d6be360e1809e86bba, role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:19.839705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:19.864144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:19.906694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be013424-5281-4a82-bdc5-55f7ac1e4a6b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:19.949451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:19.950476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:19.951139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:19.951606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:19.952560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:19.953176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:19.954777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:19.960594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 208/411] 10.222.0.60 () {58 vars in 1442 bytes} [Fri Sep 9 11:25:19 2022] PUT /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a/groups/e09314df6c2946d6be360e1809e86bba/roles/28d83235f01b49b98d223883c8317b87 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:19.965178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 204/412] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:19.970865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:20.098762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ebfee-e080-44f8-beca-0e246276fc47 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:20.140810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:20.148367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:20.149028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.149309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.149581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] PATH_INFO: `/v3/groups/44f2945b1891407aaba87e9037ca43cf` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.156114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] RBAC: Authorizing `identity:get_group(group_id=44f2945b1891407aaba87e9037ca43cf)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:20.160875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98673f6a-c53d-41c2-a70e-e9fa17f38488 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:20.163205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 209/413] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:19 2022] GET /identity/v3/groups/44f2945b1891407aaba87e9037ca43cf => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:20.168639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:20.298879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff765620-03f1-429c-b0c7-ec016abf8375 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:20.333425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:20.335972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62892b-ee2c-4a07-b608-e6fbb74d3805 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.336320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62892b-ee2c-4a07-b608-e6fbb74d3805 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.336686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62892b-ee2c-4a07-b608-e6fbb74d3805 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.342799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 210/414] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:20.343813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7747-a248-476e-87e3-67155d928431 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.344193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7747-a248-476e-87e3-67155d928431 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.344554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7747-a248-476e-87e3-67155d928431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.362154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:20.362154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.365232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.365232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] PATH_INFO: `/v3/projects/e4bd719a3a8247ec9289af5d26132989` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.368759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] RBAC: Authorizing `identity:get_project(project_id=e4bd719a3a8247ec9289af5d26132989)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:20.369372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:20.371484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 205/415] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:20 2022] GET /identity/v3/projects/e4bd719a3a8247ec9289af5d26132989 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:20.383591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:20.388447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f75c7747-a248-476e-87e3-67155d928431 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:20.389645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f75c7747-a248-476e-87e3-67155d928431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:20.470600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 211/416] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:20.543041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd7417-920b-49e8-b5ee-3ad6e1f5e280 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:20.580885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:20.581467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.581744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.582018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] PATH_INFO: `/v3/projects/e4bd719a3a8247ec9289af5d26132989/groups/44f2945b1891407aaba87e9037ca43cf/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.583927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4bd719a3a8247ec9289af5d26132989, group_id=44f2945b1891407aaba87e9037ca43cf, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:20.585988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:20.656691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2aca-2641-408a-9a72-fb2c08bc4d27 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.657315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2aca-2641-408a-9a72-fb2c08bc4d27 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.657786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b2aca-2641-408a-9a72-fb2c08bc4d27 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.659361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 212/417] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:20.664485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:20.664794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:20.665069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:20.673273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 206/418] 10.222.0.60 () {58 vars in 1442 bytes} [Fri Sep 9 11:25:20 2022] PUT /identity/v3/projects/e4bd719a3a8247ec9289af5d26132989/groups/44f2945b1891407aaba87e9037ca43cf/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:20.691383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:20.692636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:20.788868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 213/419] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:20.883956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:20.936281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:20.988994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fded501-569d-4bf1-9c7c-2c11e6d9f0f8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.026782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.027419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.027698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.027971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.038989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:21.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:21.045285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:21.046120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:21.049985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 207/420] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:21.055150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.144740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:21.200910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43294ee-f16e-41c5-9d3b-ea258a9b49e2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.213431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.237351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.237973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.238284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.238560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.239237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:21.239539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:21.241425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:21.254455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 214/421] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:21.256091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.401993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6643b5-b85e-4dd3-814d-e920bab03c76 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.418941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c90286-a402-4ea8-9b81-d78334add9d1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.441347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:21.441818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:21.463777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.464386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.464662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.464961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.485144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.485786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.486092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.486355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.506237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:21.511583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:21.512501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:21.513283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:21.514381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 208/422] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:21.522985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.565955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:21.567974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:21.725490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 215/423] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:25:21.732126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.749006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d638-e65d-4e0c-9844-03f9325e6676 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.761475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:21.790983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.791889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.792337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.792786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.793643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:21.794151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:21.795699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f0bd-5d57-41e9-a521-d2fd480f7ae5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:21.809979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 209/424] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:21.811959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794f09-2616-4708-ad65-1fd58dea6207 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.812446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794f09-2616-4708-ad65-1fd58dea6207 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.812946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd794f09-2616-4708-ad65-1fd58dea6207 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.814532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 210/425] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:21.816000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd794f09-2616-4708-ad65-1fd58dea6207 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:21.855848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f7e2-486f-405d-b683-17f9a9483eff None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.862427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:21.896881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.898381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.898996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.900234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.901599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:21.903336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f540-7c75-4d42-a746-d675ad881c2f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:21.910462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 216/426] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:21.911196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.911476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.911747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:21.931882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:21.933110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:21.941715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd794f09-2616-4708-ad65-1fd58dea6207 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:21.980818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:21.981357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:21.981637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:21.981909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.015184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 217/427] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:22.016435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.041983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.092757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.094393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.128024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57066221-15d2-4ce7-a9e7-5c090afea5d9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.164848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.166314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.166446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.166921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.167388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.168716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.170353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.177549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 218/428] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:22.179224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.272862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 211/429] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:25:22.282498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.283195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.297580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd7a75-b06c-4c23-9e6b-0abfd3f8874b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.355567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.356177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.356455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.356724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.357464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:22.357770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.359240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.373690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 219/430] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=28d83235f01b49b98d223883c8317b87 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:22.376911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.481050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b114a1f-0c73-41b3-a79f-d4f80d9267c4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.548364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ff71-ba30-4f73-b5e1-eb8502ccc937 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.569624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.577934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.579000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.586775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.586775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] PATH_INFO: `/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.586775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] RBAC: Authorizing `identity:get_project(project_id=b9e48dbb6ee64fd082bf26f66d88058b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.586775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.586775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 212/431] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/projects/b9e48dbb6ee64fd082bf26f66d88058b => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:22.592532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.599026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.599668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.599944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.600249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.618706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:22.625001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:22.625958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:22.626825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 220/432] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:22.633854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.658439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.722837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fde2c0-12d7-4bd9-94d8-3e04b0ed586d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.760210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.760895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.761177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.761449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.762280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c5e5c-40b3-41f4-a907-5c3adfc7bc1d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.762610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.764052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cea8c-7399-4b23-8ddd-da9a0e18de2c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.765812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 213/433] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:22.766779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba271-66b2-4853-ad4c-9ea103742b37 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.767061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba271-66b2-4853-ad4c-9ea103742b37 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.767390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba271-66b2-4853-ad4c-9ea103742b37 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.768778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 214/434] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:22.773864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba271-66b2-4853-ad4c-9ea103742b37 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.798126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.798899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.799127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.799600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.800242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:22.800513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.802136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98da7e1-c3ee-46cf-8008-d7d12f8226d8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.813759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 221/435] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:22.815333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.815806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.816077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.836567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:22.837815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:22.846909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.882462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba271-66b2-4853-ad4c-9ea103742b37 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:22.906676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 222/436] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:22.915709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:22.920205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:22.923069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:22.923684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:22.923964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:22.924235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:22.924954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:22.925268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:22.926733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:22.934323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 215/437] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=b9e48dbb6ee64fd082bf26f66d88058b&role.id=d3096edb08e84cd098dda68154bffd4b => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:23.037074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64460ff-c058-4034-8e6e-510602066dd9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.075742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.076641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.077064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.077481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.150086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.151760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.159992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.177603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.283208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 223/438] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:23.289919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.322583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369717ac-148a-4dc7-ba87-3f0ac82b31ef None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.360523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.367900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.368699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.368980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.369574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.383072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0ffa95b6-0573-437f-b446-fda03a8dff7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:23.387688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:23.388584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:23.389372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 216/439] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:23.391263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd3da-1f04-4bb4-be28-bc023e3338b8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.391543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd3da-1f04-4bb4-be28-bc023e3338b8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.391808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befd3da-1f04-4bb4-be28-bc023e3338b8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.393218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 217/440] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:23.398674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befd3da-1f04-4bb4-be28-bc023e3338b8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.419775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95d871-0fd3-443c-b9f6-bd28b1cd2c19 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.422104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.457478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.458206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.458491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.458770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.460481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.461914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678c64-a089-4be1-a990-c7ebe5f662f3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.463612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 224/441] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:23.464732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.465009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.465278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.485147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:23.486407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:23.522902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befd3da-1f04-4bb4-be28-bc023e3338b8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.559957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.560802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.561208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.561614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.562447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:23.562651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 225/442] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:23.562909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.563574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.567504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.577633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 218/443] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:23.583284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.650709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.678575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8443df-b23b-435b-8770-a1168e7041e7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.680372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.699130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1b55-8b2b-41e5-914a-92c05395bb9c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.719781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.720828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.721402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.721680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.722919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.724360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.729883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 226/444] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:23.731899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.735615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.736217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.736493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.736758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.809911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.811427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.842898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f0ba-c20a-4a35-b776-0acdc4503bc5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:23.866450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.879793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:23.880396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:23.881780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:23.881780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:23.885048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=d3096edb08e84cd098dda68154bffd4b, scope.project.id=e189ed93b20c4352a2f666cc5881218a) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:23.885372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:23.886849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:23.902655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 227/445] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=d3096edb08e84cd098dda68154bffd4b => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:23.902655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:23.930957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 219/446] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:23.932458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:23.936847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.015500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939361-2915-4b5d-8f4e-508b8527b2da None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.055324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.055976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.056156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa1b34-2bc5-43f3-a89e-2918a76bda53 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.056256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.056511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.070957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:24.077051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:24.078220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:24.079014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 228/447] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:24.085592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.099084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.099718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.100032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.100304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] PATH_INFO: `/v3/projects/e189ed93b20c4352a2f666cc5881218a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.103087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] RBAC: Authorizing `identity:get_project(project_id=e189ed93b20c4352a2f666cc5881218a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.105870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.110093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 220/448] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:23 2022] GET /identity/v3/projects/e189ed93b20c4352a2f666cc5881218a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:24.112265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.141366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:24.199832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:24.207948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab3a0-1fcf-43c0-8167-f1d38db0144b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.239939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9264-6c7d-4905-bea0-f8c3004a23ff None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.252981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.253637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.253918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.254330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.254893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:24.255192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.257106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.269931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 229/449] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:24.276553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.277963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.278636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.278916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.279190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] PATH_INFO: `/v3/groups/e09314df6c2946d6be360e1809e86bba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.280317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] RBAC: Authorizing `identity:get_group(group_id=e09314df6c2946d6be360e1809e86bba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.281816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.283446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 221/450] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/groups/e09314df6c2946d6be360e1809e86bba => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:24.289877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.380086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2137-36f8-4375-ba6f-d8f4550499bf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.388922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:24.398571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad544d3-441c-4c6c-b62d-f16895c80038 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.415527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.416127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.416401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.416668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.432183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.432793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.433063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.433329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.434047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] RBAC: Adding query filter params (group.id=e09314df6c2946d6be360e1809e86bba, role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=e189ed93b20c4352a2f666cc5881218a) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:24.434380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.435822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.441527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 222/451] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/role_assignments?group.id=e09314df6c2946d6be360e1809e86bba&scope.project.id=e189ed93b20c4352a2f666cc5881218a&role.id=28d83235f01b49b98d223883c8317b87 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:24.443129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:24.492858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.494398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.605176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 230/452] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:25:24.611006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.718007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982576c-a520-4c01-90e7-ab693210a1fc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.753207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.753778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.754079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.754342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] PATH_INFO: `/v3/projects/e4bd719a3a8247ec9289af5d26132989` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.755469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] RBAC: Authorizing `identity:get_project(project_id=e4bd719a3a8247ec9289af5d26132989)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.757466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.759227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 223/453] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/projects/e4bd719a3a8247ec9289af5d26132989 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:24.764900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.824659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:24.871889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82fe08-5e95-4353-883e-bd5305858f8a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:24.905882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:24.906522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:24.906797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:24.907064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] PATH_INFO: `/v3/groups/44f2945b1891407aaba87e9037ca43cf` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:24.908146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] RBAC: Authorizing `identity:get_group(group_id=44f2945b1891407aaba87e9037ca43cf)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:24.909543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba7a9b-7808-4439-9635-bcba985f3485 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:24.911113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 231/454] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/groups/44f2945b1891407aaba87e9037ca43cf => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:24.917575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:24.942993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:25.016007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b3527-1e1d-4e65-8aed-43cf3757309b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:25.049123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:25.049716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:25.049994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:25.050292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:25.051009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] RBAC: Adding query filter params (group.id=44f2945b1891407aaba87e9037ca43cf, role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e4bd719a3a8247ec9289af5d26132989) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:25.051315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:25.052739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469097fd-dff9-483e-85e4-3913a4fcc2ad None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:25.058410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 224/455] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Sep 9 11:25:24 2022] GET /identity/v3/role_assignments?group.id=44f2945b1891407aaba87e9037ca43cf&scope.project.id=e4bd719a3a8247ec9289af5d26132989&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:27.800952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446252-9b69-48d0-8f35-95e26fd2a02e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.801916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446252-9b69-48d0-8f35-95e26fd2a02e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.802455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446252-9b69-48d0-8f35-95e26fd2a02e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.804716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 232/456] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:27.814425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ddf61-14b2-49c5-8452-cb2530f487fc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.814425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ddf61-14b2-49c5-8452-cb2530f487fc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.814425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ddf61-14b2-49c5-8452-cb2530f487fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.839385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5a4ddf61-14b2-49c5-8452-cb2530f487fc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:27.841083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5a4ddf61-14b2-49c5-8452-cb2530f487fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:27.932692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 225/457] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:27.963993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150c61-af35-4c51-856e-3163ca4ed696 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.964589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150c61-af35-4c51-856e-3163ca4ed696 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.965061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150c61-af35-4c51-856e-3163ca4ed696 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.966886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 233/458] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:27.971778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.972072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.972348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.977515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f375f2-9cee-4812-9735-fa73c304783a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.978043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f375f2-9cee-4812-9735-fa73c304783a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.978509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f375f2-9cee-4812-9735-fa73c304783a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.980240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 234/459] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:27.985334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5567c-f1a0-4e9d-bc91-ae4870cb3d53 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:27.985854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5567c-f1a0-4e9d-bc91-ae4870cb3d53 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:27.986375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5567c-f1a0-4e9d-bc91-ae4870cb3d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:27.998789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:28.001449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:28.022094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c9e5567c-f1a0-4e9d-bc91-ae4870cb3d53 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:28.022094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c9e5567c-f1a0-4e9d-bc91-ae4870cb3d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:28.026098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.128127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 226/460] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:28.142295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 235/461] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:28.236334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.315145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb931-301a-47b7-99e3-2f1939e1fbcb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.315464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb931-301a-47b7-99e3-2f1939e1fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.315700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb931-301a-47b7-99e3-2f1939e1fbcb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.317158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 236/462] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:28.327574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.327851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.328119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.359036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:28.362485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:28.362606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbd56f-7b22-4913-9e88-3bd9e1fadf2e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.400823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.401563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.401847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.402145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.413490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.414967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.422588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:28.432529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:28.433423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:28.434371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:28.434371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.434371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:28.434371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 227/463] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:28.440873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.466992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.496586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 237/464] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:28.550143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a3c3f-9598-4c1e-8884-56aabf479285 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.558854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.588231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.588946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.589226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.589500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.590203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:28.590510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.591990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.610973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 228/465] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:28.611995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.671807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.676602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5fb9-429e-4931-b598-2e15b510cec8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.713741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.714412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.714690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.714966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.715345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0566244-a6bb-4498-83e3-9c7d03c8c386 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.726752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.727632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:28.744595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:28.746773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:28.748845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:28.748845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.748845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:28.748845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 238/466] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:28.748845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.758040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.758708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.759050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.759319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.769543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.771012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.786131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:28.786360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:28.788454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:28.790565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:28.790565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:28.790565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:28.790565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 229/467] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:28.790565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:28.861882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbd2f5-0755-478a-9806-8838cf0235b1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.899490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba149fb-4cf6-42ab-af23-a77175f7bdb6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:28.903519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.904413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.904884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.905168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.905821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.907656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.910205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:28.935114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:28.935646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:28.935922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:28.936294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:28.937233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:28.937540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:28.938938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:28.952437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 230/468] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:28.953134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.026292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 239/469] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:28 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:29.026292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.070330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1c190-34d7-447b-8229-c2fe4995bf76 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.118959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.120192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.120539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.121086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.121751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:29.122086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.124215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.131118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:29.141423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 231/470] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:29.162298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.162462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d82eb9-d807-4a36-8050-8e13924db596 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.186399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.186978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.188085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.188363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.192860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:29.205918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.207424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:29.224895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:29.225865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:29.226785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:29.226785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:29.226785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:29.226785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 240/471] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:29.227238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.290746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca8cc3-1a51-46c7-bac0-2144ffa070a5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.333421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.334623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.334906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.335177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.335828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.337304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.371545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac61d49-e663-4db3-b11d-e4e160923e64 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.406073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.406698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.406977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.407249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.407892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.409319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a4903-dbde-4a0d-9253-4cd1cc18183e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.424967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 232/472] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:29 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:29.424967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.489653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:29.502286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 241/473] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:29 2022] POST /identity/v3/users => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:29.502286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b652af-297e-43d6-9724-2deeca08df92 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.502286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b652af-297e-43d6-9724-2deeca08df92 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.502286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b652af-297e-43d6-9724-2deeca08df92 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.502286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 242/474] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:29.510624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.510951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.511181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.551952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:29.553286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:29.576264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:29.602197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8c3a-259f-49b7-9a46-e9c2df071b8b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.657082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.657714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.657995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.658819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.659496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:29.660100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.661817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.676165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 243/475] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:29.679010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 233/476] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:29.689631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:29.795343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ad90f-22dd-4dba-8834-95dc277532d1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:29.827918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:29.834833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:29.840620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:29.840620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:29.840620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:29.840620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:29.840620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:29.934564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 244/477] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:29 2022] POST /identity/v3/users => generated 249 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:29.967377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:30.110579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a50f5-28f9-4ff8-84f2-69405bc9be95 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:30.150012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:30.150651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:30.150925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:30.151190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:30.151875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:30.153245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:30.200442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.sql.core [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:25:30.200442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:25:30.200442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [parameters: {'id': 'a06fac4d4d75445db4e36a8b1f479783', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:25:30.200442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81665) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:25:30.217256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:25:30.218187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application yield resource Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise self.value Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application session.commit() Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:25:30.219113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:25:30.220010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:25:30.220921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:25:30.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [parameters: {'id': 'a06fac4d4d75445db4e36a8b1f479783', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:30.225960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:25:30.235602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:30.236695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 234/478] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Sep 9 11:25:29 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:30.236784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:30.341710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf34d5-f74f-48dd-9ad5-3f50910e915e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:30.377562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:30.378421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:30.378899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:30.379409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:30.390164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-54929df8-b681-4553-93e7-52c312a1da0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:30.395536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:30.397096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:25:30.398081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:30.400215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 245/479] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:25:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:30.409158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:30.422480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:30.549459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5892c0b-bf18-4d57-8a5b-8bc90c57b04e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:30.601258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:30.602260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:30.602737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:30.603202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:30.604064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:30.604558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:30.606248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc48de-4fc8-4ac3-b589-17f181423ffa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:30.617847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 235/480] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:31.796916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3240-7c12-4b29-b347-4821bebce281 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:31.797400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3240-7c12-4b29-b347-4821bebce281 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:31.797400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3240-7c12-4b29-b347-4821bebce281 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:31.798987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 246/481] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:31.804837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e603-b8a9-40dd-ac6c-cebec9c301fa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:31.805260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e603-b8a9-40dd-ac6c-cebec9c301fa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:31.805432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363e603-b8a9-40dd-ac6c-cebec9c301fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:31.827932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e363e603-b8a9-40dd-ac6c-cebec9c301fa None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:31.829117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e363e603-b8a9-40dd-ac6c-cebec9c301fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:31.901844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 236/482] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:32.228932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68049-5383-486a-a802-d024e85447c9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.229330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68049-5383-486a-a802-d024e85447c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.229511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68049-5383-486a-a802-d024e85447c9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.231105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 247/483] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:32.236118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767761d-4853-4b93-89c7-0fd4982e5bb3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.236404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767761d-4853-4b93-89c7-0fd4982e5bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.238041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360567-4dea-4346-8278-7ecceccac9cf None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.238344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360567-4dea-4346-8278-7ecceccac9cf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.238612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360567-4dea-4346-8278-7ecceccac9cf None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.240044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 248/484] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:32.240605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66360567-4dea-4346-8278-7ecceccac9cf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:32.241031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767761d-4853-4b93-89c7-0fd4982e5bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.250776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:32.266670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a767761d-4853-4b93-89c7-0fd4982e5bb3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:32.275677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a767761d-4853-4b93-89c7-0fd4982e5bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:32.300552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:32.352002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 237/485] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:32.354909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.355191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.355459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.381838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66360567-4dea-4346-8278-7ecceccac9cf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:32.396512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:32.400574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:32.441863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:32.442538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.442860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.443125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.464730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:32.482504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:32.483416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:32.484212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 249/486] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:32.495406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:32.506167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 238/487] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:32.678089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd8695-8add-45a1-807b-e544977fa3ca None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:32.681450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:32.718607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:32.719216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.719496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.719767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.720445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:32.720746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:32.722138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322cde-9731-4e52-bcfd-ae05b7ec4b3f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:32.735453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 250/488] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:32.741086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:32.765404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287858fb-5e6f-4a32-bb29-11533d57c9ff None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.765685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287858fb-5e6f-4a32-bb29-11533d57c9ff None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.765964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287858fb-5e6f-4a32-bb29-11533d57c9ff None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.767717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 251/489] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:32.772571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:32.772861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:32.773139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:32.834694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:32.836013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:32.887606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:32.918757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 252/490] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:32.919738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:32.956677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e163-f38f-4578-947b-70c12cc4e73f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.026097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.027238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.027724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.028320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.047987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.050649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:33.087525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.099038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:33.099880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:33.099880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.099880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:33.099880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 239/491] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.099880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.100152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.112360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae516514-feb8-40bc-804a-4cea615c3708 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.153653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.154304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.154618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.154895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.165195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:33.169912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.170940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 253/492] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.171821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.261329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fa35d-fa76-4b96-a2ad-46e0d95affee None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.275560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.285291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa00996-74f4-4a9b-8c2f-4255191df20c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.295268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.295912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.296199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.296474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.308624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:33.314041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.315025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:33.315854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 240/493] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.316509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.333042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.334411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.334692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.334963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.335618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:33.336000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.337409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.354115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 254/494] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:33.354115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.380926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.432768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ee80-7b5b-49be-ba2a-236f6fb2f9cf None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.467302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481705e4-1f7a-40c7-b01a-706ed4aa9ac7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.467686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.468309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.468585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.468847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.469510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:33.469810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.471205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.481919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 241/495] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:33.483503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.503995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.504615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.504890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.505199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.515963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:33.520903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.521802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 255/496] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.522749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.523889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.588453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ca4b-4a68-4f29-9492-b1e48d4514e9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.601965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.622315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.622909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.623183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.623474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.623795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d39bf-e0db-4884-ac14-991839b4dbfb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.624122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:33.624421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.625797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.633761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 242/497] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:33.634562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.662109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.662725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.662996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.663260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.670178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.671696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-05b20b01-44d5-4431-98d7-c5fd3b3444e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:33.685438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:33.687345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 256/498] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.687891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15248de-1b40-4b37-ad76-82475e0d4bd0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.687955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15248de-1b40-4b37-ad76-82475e0d4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.688175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15248de-1b40-4b37-ad76-82475e0d4bd0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.689498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 257/499] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:33.690065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15248de-1b40-4b37-ad76-82475e0d4bd0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.735586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.739075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72456e2-5a0b-406d-b656-e6a11073bfcf None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.774356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.774974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.775252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.775520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.786159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.787601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.794356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15248de-1b40-4b37-ad76-82475e0d4bd0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:33.801231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.802232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:33.803098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:33.803098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.803098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:33.806196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 243/500] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.807182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.827751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.828337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.828608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.828874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.829528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:33.829821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.831212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.836984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.840812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 258/501] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:33.841705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:33.904592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53193e-39cf-44b9-aa64-60c73fc5f476 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.937617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.938267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.938547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.938818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.939367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93dba4-901a-4fc4-ba7e-b599a850bf76 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:33.949262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.950747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-cc935cf6-49b9-427d-b3cc-3c47ef52ee69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:33.965498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:33.966536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:33.967555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:33.967555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:33.967555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:33.967555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 244/502] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:33.968131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.968358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.968629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.976283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:33.979426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:33.980025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:33.980296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:33.980564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:33.981370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:33.981726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:33.983334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:33.991831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:33.993073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:34.022115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 259/503] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:34.022988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.065730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 245/504] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:34.066602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.084992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.127854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4bd01-5f1b-4144-845c-ab80b00ebfdb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.166733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.167345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.167622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.168055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.168979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:34.169289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.170692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.183024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 260/505] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:34.183821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.185147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587bba5-0b81-4ce7-9a1d-930842567e15 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.217177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.221876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.222527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.222802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.223066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.233184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.234727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:34.249356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:34.250300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:34.251183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:34.251183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.251183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:34.251183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 246/506] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:34.251409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.293983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b1e4d-45ed-4c05-a4a5-498294cd6acd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.313144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.330523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.331114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.331518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.331788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.332426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:34.332726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.334689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.347048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 261/507] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:34.347857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.368826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76395e2-390d-4127-9a94-f51a441d569c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.404125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.404731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.405005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.405275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.449334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735d9a-0380-4bf0-829e-fc4d2785de43 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.455680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.471018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.472483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.482368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.482990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.483318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.483589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.490899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.492296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:34.503012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:34.503896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:34.504791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 262/508] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:34.505890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.543347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.571429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 247/509] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:34.572352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.601573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc99e-8bac-40af-8d7b-ad22b94632a5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.635484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.636011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.636296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.636567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.637197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:34.637498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.638878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.650845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 263/510] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:34.651808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.654482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.669479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216cb7-67fe-474e-8cc5-b7f24f44e06b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.703272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.703883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.704156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.704421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.713914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.715382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:34.729674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:34.730627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:34.731514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:34.731514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.731514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:34.731514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 248/511] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:34.732021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.750007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.756862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a401-c8d3-4726-b8ed-872c2f308424 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.789286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.789917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.790376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.790651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.798110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.799564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:34.811090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:34.811948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:34.812828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 264/512] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:34.813371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.835709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a7ff0-4f24-49b1-b01c-715ced8a7b01 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.868984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.869582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.869861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.870166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.870885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:34.871197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.872614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.883847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:34.908764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 249/513] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:34.909461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470cf0d-c970-478e-8cfb-c8f4561460db None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:34.909837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.945722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:34.946369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:34.946644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:34.946911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:34.956660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:34.958179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:34.972370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:34.973427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:34.974816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:34.974816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:34.974816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:34.974816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 265/514] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:34.976819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:34.989131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.017977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fb289-317d-4ad3-be7b-403c0c4f2d1a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.051887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.053130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.053405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.053669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.060425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.061871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:35.074720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:35.075590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:35.076698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 250/515] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:35.077764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.082673 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e1c98-2914-4041-a9f5-679a7f9ee10f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.116542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.117141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.117417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.117685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.118349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:35.118651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.120010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.131549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 266/516] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:35.132334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.133369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.176660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd7790-3c05-46e1-9f18-4aa6918a94a2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.204725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.210208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.210838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.217394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.217394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.217394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:35.217394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.217394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.231376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 251/517] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:35.232173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.234501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fcab3-11c6-4bc8-8562-30c7dcc23d03 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.268340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.268937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.269215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.269494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.329370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c76ec-dcc4-4a80-bfd6-70be346d390f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.336537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.338478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.342863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.363086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.363690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.363969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.364239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.365424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:35.365727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.367120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.379550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 252/518] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:35.380402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.416199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.448732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 267/519] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:35.449467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.483054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af348b96-2232-470a-a1f6-b97a2588f134 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.515276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.515879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.516152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.516425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.517137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:35.517440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.518895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.536190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 253/520] 10.222.0.60 () {56 vars in 1139 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:35.536986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.545729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109812da-4223-4a03-8b62-44006f1bb9ae None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.566204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.579723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.580325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.580604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.580875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.581517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.582983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.634201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ef7c4-0f0d-4d9e-89b6-b541c8ce48ab None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.666011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.666630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.666904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.667173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.705436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 268/521] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:35 2022] POST /identity/v3/users => generated 247 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:35.706644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.739053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.749997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.749997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.777059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:35.826636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416f4d9-fbc4-4c9d-9a4f-04b41f1ba93a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.857128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 254/522] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:35.858458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.870733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:35.871530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:35.871810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:35.872084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:35.879267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:35.881259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:35.895775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:35.896736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 269/523] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:35.897582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:35.975143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d77812-f9cd-4970-ba59-bbb92ac2a774 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:35.992947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.004533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.009888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961080ce-9869-400a-a23b-893e218ce344 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.011200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.011830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.012107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.014844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.023165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.024612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:36.038887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:36.044745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:36.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:36.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:36.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 255/524] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:36.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.051703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.052315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.052699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.052977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.134703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.136420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.144300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5172-b6be-43eb-98b8-e68b872e5f82 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.180110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.180736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.181011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.181288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.192293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.193775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:36.208267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:36.209751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:36.210975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:36.210975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.210975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:36.210975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 256/525] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:36.210975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.236183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.248412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.257409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 270/526] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:36.258533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.319293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e662b5-8350-497c-9a51-06ff51dcac22 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.354310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.354912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.355190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.355462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.356142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:36.356448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.357895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.367245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e8f4b-d451-40e9-a527-5424a42c14a6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.379929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 257/527] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:36.379929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.414228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.414833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.415109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.415383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.416021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:36.416323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.417678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.433637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 271/528] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:36.433637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.481232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.484640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.491735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b0ef8-dd77-48dd-88a3-9dda490fd826 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.530392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.531012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.531376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.531568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.532202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:36.532504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.533888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.540728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac2663-3256-406b-988d-76519c9a46f6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.549558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 258/529] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:36.549558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.579836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.580590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.580873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.581143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.593310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.595034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:36.609122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:36.610024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:36.610944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:36.610944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:36.610944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:36.610944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 272/530] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:36.611159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.659123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f748-7d7d-4a35-8207-efb06f14c9a2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.694253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.694857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.695398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.695667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.696383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=10d651db5bf74166bff093d317d83a6c) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:36.696691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.698140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.714087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 259/531] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/role_assignments?user.id=10d651db5bf74166bff093d317d83a6c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:36.714087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.732150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.734876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:36.737418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90910-bc7d-483b-81d3-bb23ec089e3c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.790405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.790995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.791276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.791551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.800725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.802269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:36.815739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:36.816693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:36.817547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:36.822449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 273/532] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:36.823240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.843668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc172f-e4ee-42c2-8fa0-e57b47207bea None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.898891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.899827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.900285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.900765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.910308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.912156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:36.929346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:36.930287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:36.931122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:36.933943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 260/533] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:36.934892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:36.947387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f654-382b-41f6-817e-3e7a0b410c97 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:36.990283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:36.991450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:36.991736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:36.992118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:36.993555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:36.994124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:36.997385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.012619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 274/534] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.018726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.033347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:37.050494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:37.060373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46271f92-7c99-47a8-a4b4-01d8132f16b6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.105972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.106756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.107188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.107467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.108938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.109249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.110718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.126511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 261/535] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.127341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.147813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167853d4-4671-4b73-a7a8-05adfe042332 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.192071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.192779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.193060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.193422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.194906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.195479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.196910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.215170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 275/536] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.216259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.247630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0557d5-6fda-4e60-9592-653acbff0ee1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.285720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.286468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.286856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.287167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.294407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.295975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.297385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:37.298694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:37.310269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:37.311299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:37.312322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:37.318170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 262/537] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:37.319699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.321142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864dd8d-8026-448d-b4b2-5cd48f7213bd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.357477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.358196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.358379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.358666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.359388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.359765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.361431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.374089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 276/538] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.374089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.511664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0436a7-1a39-4025-a16d-fc21c777ae91 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.565193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.566046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.566370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.566644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.569061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=2eccd6d01b9a48cf946ea37543cf8c3a) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.569061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.569661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.576654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 263/539] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/role_assignments?user.id=2eccd6d01b9a48cf946ea37543cf8c3a&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.597383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e26c-0a20-471f-a0e4-d403449f20a7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.638593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.639716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.640226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.640706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.641694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.642269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.644032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864369f-2c36-4aee-8e83-850c7b87bfc6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.659249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 277/540] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:37.666256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:37.760518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:37.812664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744759-4750-4fc2-81a6-137b301cba8b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:37.849565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:37.851145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:37.851640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:37.852134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:37.853077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=b062fa227cd043719c7eff9d2b7e4bf0) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:37.853584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:37.855274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d263e-1941-4904-8410-559685aaf2f0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:37.861726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 264/541] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:37 2022] GET /identity/v3/role_assignments?user.id=b062fa227cd043719c7eff9d2b7e4bf0&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:38.015668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208888-a56d-4c1d-a8b4-87221e80d889 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:38.016677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208888-a56d-4c1d-a8b4-87221e80d889 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:38.017208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208888-a56d-4c1d-a8b4-87221e80d889 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:38.019270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 278/542] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:38.025228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:38.025820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:38.026328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:38.049585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:38.051699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:38.141925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 265/543] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:38.432842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208888-a56d-4c1d-a8b4-87221e80d889 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:38.436191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:38.558800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208888-a56d-4c1d-a8b4-87221e80d889 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:38.599075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:38.599758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:38.600098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:38.600369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:38.616229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:38.621350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:38.622240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:38.623047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 279/544] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:38.628926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:38.705563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:38.764746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf00b86-0a6a-47da-b061-352882797a35 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:38.819777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:38.820644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:38.820938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:38.821242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:38.821913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:38.822238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:38.823606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:38.832391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 266/545] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:38.837927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:38.961423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:39.008375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88bc39-1a3f-435c-81c5-1cfe7ebe25af None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:39.047204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:39.047829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.048091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.048359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.058920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:39.060373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-73f276a2-0a7c-45bd-8eeb-879c3f95fedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:39.082378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:39.083423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:39.084290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:39.084290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:39.084290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:39.084290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 280/546] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:39.089362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:39.264927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4ea1f-e58c-4741-954d-2809f440bc09 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:39.288223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:39.317532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:39.318165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.318445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.318815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.319574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:39.319778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:39.321414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:39.328475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60583-855a-443d-803e-5e8babd85cb5 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.328840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60583-855a-443d-803e-5e8babd85cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.329194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60583-855a-443d-803e-5e8babd85cb5 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.330898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 281/547] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:39.336622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.336977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.337331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.338100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 267/548] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:39.343368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:39.356197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:39.357500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:39.495535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 282/549] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:39.536630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a54-44e4-4252-b32a-4fcada5a3f5b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:39.580996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:39.581623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.581901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.582204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.595782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:39.597247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:39.612611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:39.613571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:39.614515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 268/550] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:39.619619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:39.688719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:39.750702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa0630-e525-4563-ba68-414a84e1feb7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:39.792944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:39.793488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:39.793766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:39.794109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:39.794851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:39.795155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:39.796577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:39.818281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 283/551] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:25:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:39.825381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:39.879529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:39.955852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9342d-6091-4314-b918-112a6458e1db None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.002978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.003619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.003896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.004171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.014253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.121848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.124155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31b940-77d7-44a1-ad72-ceb1f527df61 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.184583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.187716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adf8e3-ea8f-48f8-9f06-30cad12c5119 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.191077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.229435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.230760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.231888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.232398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.246892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-481bdcc2-cd9b-4238-94f2-d03c7b7ddd05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:40.253309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:40.254362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:40.255184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:40.258728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 284/552] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:40.260198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7463-597b-4e58-b899-2ca096761a21 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.260613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7463-597b-4e58-b899-2ca096761a21 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.261012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7463-597b-4e58-b899-2ca096761a21 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.266596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 285/553] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:40.268197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f7463-597b-4e58-b899-2ca096761a21 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.278216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 269/554] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:40.278216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.278216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.278216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.295222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:40.296426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:40.378644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 270/555] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:40.379751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.399298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f7463-597b-4e58-b899-2ca096761a21 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.441799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.442460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.442743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.443016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.443706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:40.444042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.445446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506a4f7-4dad-4706-ab7d-b8f02042f740 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.455881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 286/556] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:40.457237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b966b-8a01-4dfe-9861-8d6845fbe058 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.457524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b966b-8a01-4dfe-9861-8d6845fbe058 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.457800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b966b-8a01-4dfe-9861-8d6845fbe058 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.459535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 287/557] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:40.461778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b966b-8a01-4dfe-9861-8d6845fbe058 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.480799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.494001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.529196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37c066-92ec-494c-abf7-2a1fc483ea70 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.571071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.571699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.571978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.572244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.579418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b966b-8a01-4dfe-9861-8d6845fbe058 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.582260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.583702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a0053ca0-cdac-440b-8783-cf878c4d31c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:40.599292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:40.600283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:40.601193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:40.601193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:40.601193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:40.601193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 271/558] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:40.601862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.602476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.602476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.620206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:40.620730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.621356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.621460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:40.621911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.621911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.632485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.633855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-16e19a7f-6d59-4d82-a21b-7ec80c10f79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:40.650038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:40.651027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:40.651898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:40.651898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:40.651898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:40.651898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 288/559] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:40.652845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cddd0-3b7e-4d9c-9198-ab7c30814410 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.653128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cddd0-3b7e-4d9c-9198-ab7c30814410 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.653402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cddd0-3b7e-4d9c-9198-ab7c30814410 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.658088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 289/560] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:40.658088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cddd0-3b7e-4d9c-9198-ab7c30814410 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.698421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 272/561] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:40.699020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.734465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.750272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:40.758628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cddd0-3b7e-4d9c-9198-ab7c30814410 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.792749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.793352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.793632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.793901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.794568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:40.794869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.796224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb4e31-2aa5-4f0f-bf16-5a3167a2ce97 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.799848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96068d-46be-4be3-ad40-14e6ddb389de None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:40.812469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 290/562] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:40.812469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.812469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.812469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.827589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:40.828770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:40.839866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:40.840576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:40.840916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:40.841258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:40.841964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:40.842300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:40.843671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:40.859676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 273/563] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:40.860588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.896527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 291/564] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:40.906963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:40.980804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.001831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506fe6d-4cfd-4e90-a279-960cc44313e2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.013976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.022518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55054d2-c222-49d2-a7c2-80a0e8095498 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.036876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.037509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.038424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.038687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.048888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:41.053747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.054896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:41.055860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 274/565] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.056727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.086502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.087533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.088026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.088504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.096250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.097721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:41.109204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.110154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 292/566] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.111066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.157650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04544495-09fe-47a8-9fc8-1fdf6d1366c0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.189626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.190277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.190554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.190826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.198917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.200375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.210587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf461a8-f90c-4d20-b7bf-22d56939d68a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:41.211355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.212317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 275/567] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.213291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.248515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.249156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.249428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.249692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.259981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:41.265172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.270197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 293/568] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.271304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.301192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.315958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967a63b-9ac4-438a-ba3f-10ed8ffaa8f0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.336943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.352887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.353486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.353756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.354132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.354794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.355087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.356572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.367559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1b2db-700d-4126-9a2c-bf310d04a809 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.371284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 276/569] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.371284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.401384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.402015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.402323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.402598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.403280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.403582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.405001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.416730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 294/570] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.417370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.463026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.465495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.473784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6c2e5-5463-4c8c-97ca-947c197e24aa None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.505706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.506331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.507200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.507482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.508202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.508504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.509927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.515834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42ba90-5159-4e1f-abe8-186daf619c1a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.527155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 277/571] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.527968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.549080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.549700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.549975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.550273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.560518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:41.565269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.566411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 295/572] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.567242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.627537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29735f8-c1db-4d9f-b973-69fbe34e3ba8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.660463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.661061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.661337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.661609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.662295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.662595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.663664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4f554-4984-4015-a3b4-2d54462a3396 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.664019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.670834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.671809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 278/573] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.672446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.686912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.702980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.703600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.703877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.704145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.715086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.716620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:41.731512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.732519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:41.733420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:41.733420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:41.733420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:41.733420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 296/574] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.733420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.776180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc4426-57c8-44dc-a1c5-84ab064046ea None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.808659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.809479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.809594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.809785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.810531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=da9b976e48d3468b82d9168efced0b68) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.810844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.812313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.817579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 279/575] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/role_assignments?user.id=da9b976e48d3468b82d9168efced0b68&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.818380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.841888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d234e1-2048-4cdf-b3a3-6dfc7f769979 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.881010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.881691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.882103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.882494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.894014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.895597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:41.913730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:41.914928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:41.916033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:41.916033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:41.916033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:41.916672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 297/576] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:41.917118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.917500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:41.931534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:41.945231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef257-e764-45c2-a34c-800e3f0dfa36 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:41.983391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:41.984011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:41.984283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:41.984611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:41.985289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:41.985593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:41.987015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:41.996210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 280/577] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:41.997051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.035822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a28c4-dedf-4c0c-98ff-ef7102da5b79 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.077922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.078601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.078882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.079154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.092556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.094013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.106364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ef26-03ff-4fc5-a10e-9f4d027848aa None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:42.108275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:42.109351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:42.110553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:42.110553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:42.110553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:42.114011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 298/578] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:42.114866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.146677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.147345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.147627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.147901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.148553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.148855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.150272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.158164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.163037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 281/579] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.167946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.182778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.225458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931372-c3b6-4e46-b17d-57759c3e5866 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.265607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.266387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.266769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.267137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.267893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.268293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.269740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.270364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5007e3a-2d7a-41d9-a678-d33ef0a63211 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.283410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 299/580] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.284553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.310969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.311684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.312029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.312532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.323158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.324362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:42.344366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:42.345595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:42.347084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:42.347084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:42.347084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:42.351906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 282/581] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:42.352629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.400729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9576-577d-4af8-b79d-c17986b534b9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.408625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.425508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.437068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.437693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.438089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.438376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.439021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.439327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.440684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.452855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 300/582] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.453680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.468222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3ae2b-e9ae-46ac-9a77-3ddfd51872d6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.506464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.507636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.507909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.508178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.515898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.517332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:42.528706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:42.529897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:42.531151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 283/583] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:42.535140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.563906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9483cb-799a-4fb6-9e56-15289ad155d0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.600349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.601731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.602188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.602726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.609899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.611555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:42.623674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:42.624749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:42.625867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:42.628178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 301/584] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:42.628966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.638857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da73c2b-6deb-48a1-a68a-df30213bbbb2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.651086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.677275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.679976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.681195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.681493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.681767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.682856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.683187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.685111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.705941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 284/585] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.705941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.750831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de7260-0682-4efd-b37d-8ccb0d33264c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.787703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.789051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.789422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.789776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.798434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.799964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.808454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff14c7b-e992-4fe4-860f-4160bb8afb92 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:42.811566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:42.812497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:42.813459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:42.817329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 302/586] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:42.818189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.845137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.845732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.846134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.846307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.847031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.847337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.848773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.867040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 285/587] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.867040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:42.899935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.929576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5829-79f8-4712-b107-8ca10ede7c8a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.931343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:42.965909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:42.966607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:42.966982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:42.967341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:42.968100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:42.968469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:42.969957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:42.975240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf055fb-c6c2-45ce-ae70-37993767c33f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:42.987016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 303/588] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:42.987016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.015727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.016966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.016966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.016966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.024532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.025973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:43.043111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:43.044297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 286/589] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:43.048559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.107949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dda9c-2bb8-41d3-84ee-380be276c1a7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.142367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.143888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.144483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.144765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.145041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.145769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:43.146097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.148097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.155500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97db9d-2e90-44d0-ab1e-31eee90f252a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.170935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 304/590] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:25:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:43.170935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.193591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.194266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.195064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.195340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.195446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.206362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.208025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:43.222511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:43.223608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:43.224651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:43.224651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.224651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:43.224651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 287/591] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:43.224651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.283629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36113c8-47af-4411-af0d-9871ae8c1ed4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.320646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.321553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.321842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.322145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/10d651db5bf74166bff093d317d83a6c/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.329111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321162-3af6-47fc-8eac-b3f9f9b2f686 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.336872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=10d651db5bf74166bff093d317d83a6c, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.338333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.362734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.372703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.379246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.387151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 288/592] 10.222.0.60 () {56 vars in 1139 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:43.388006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.436978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 305/593] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:43 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/10d651db5bf74166bff093d317d83a6c/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:43.447402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.506106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41300192-c080-48b6-99f8-531024bb43e9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.546043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.546992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.547460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.547877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.549599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.561625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.563138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:43.580954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:43.582426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:43.583307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:43.583307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.583307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:43.587122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 289/594] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:43.588029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.599408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d788f-948c-4e2e-92b8-6a05e9357419 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.646535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.647546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.648601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.648899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.649555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:43.649859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.650201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.651263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.666127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 306/595] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:43.667274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.720146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3753b-37fe-4f26-bb2e-ff1a282a1d70 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.755164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.755790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.756066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.756393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.767574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.769085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:43.784817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:43.785824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:43.786752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:43.786752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:43.786752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:43.786752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 290/596] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:43.787407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.845797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0ae49-dc4c-4d3b-b830-8ca2bdc51c47 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.892220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.899600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f9b2e-899a-493e-b78f-10b324a88abe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:43.901511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:43.903780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.904664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.905165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.905663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.906549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:43.907071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.909316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.926065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 307/597] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:43.927308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:43.939576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:43.940351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:43.940586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:43.940913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:43.948423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:43.949923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:43.962145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:43.963229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:43.964101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 291/598] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:43.964593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.051921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ce242-15c1-434e-8c4e-2499e1f750da None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.069154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e1ea5-44d0-4da6-8680-2e1b82990cc5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.087951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.088801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.089250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.089688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.090510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:44.090960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.092463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.106151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 308/599] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:44.107161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.119335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.119980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.120256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.120526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.127675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.129126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c589e93e-9a67-4349-8c36-cfa2e0dee4be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:44.140570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:44.141453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:44.142427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 292/600] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:44.142932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e7e8-8bd6-4db5-a990-351063985a9c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.143315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e7e8-8bd6-4db5-a990-351063985a9c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.143315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e7e8-8bd6-4db5-a990-351063985a9c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.146129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 293/601] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:44.146129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e7e8-8bd6-4db5-a990-351063985a9c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.173102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.175523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.229371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be6d3-b453-4eca-9f4b-16bd3f46896b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.258673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e7e8-8bd6-4db5-a990-351063985a9c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.266111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.266721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.266999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.267270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.267958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:44.268260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.269667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.282278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 309/602] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:44.283089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.301259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.302200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.302638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.303053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.309994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.311649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ce71407f-f679-4a06-980c-f0f1b4373305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:44.324483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:44.325543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:44.326611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:44.331950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 294/603] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:44.331950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.331950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.331950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.348436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:44.349620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:44.406630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4d6e-bfde-40bb-98d9-aa2402460c6a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.424725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.435714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 295/604] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:44.437571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.449319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.450422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.450900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.451360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.452280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:44.452954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.455036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.457581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.468350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 310/605] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:44.469493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.553261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cb06b-575a-4383-819f-dee92db91950 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.573182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273d99-cc35-4bc9-8207-a53dcd470413 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.590625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.591351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.591759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.591931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/2eccd6d01b9a48cf946ea37543cf8c3a/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.609135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=2eccd6d01b9a48cf946ea37543cf8c3a, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.609715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.610598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.610814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.611350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.611813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.612680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:44.613160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.614792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.628452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 311/606] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:44.629027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.661696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.713622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 296/607] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:44 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/2eccd6d01b9a48cf946ea37543cf8c3a/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:44.714889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:44.735089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe246e-7ad2-4063-9e9b-c9f70a6f806d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.779769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.780459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.780842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.781127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.801707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=e7b89441837347cba3200f9104c0f16c, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.803225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.803953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:44.846570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf4737-98bc-47e7-8fb2-16a58fe6d8c9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:44.884058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:44.884761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:44.885039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:44.885311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/b062fa227cd043719c7eff9d2b7e4bf0/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:44.900526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=b062fa227cd043719c7eff9d2b7e4bf0, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:44.901993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:44.939975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 312/608] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:44 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:44.940811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.021364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 297/609] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:44 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/b062fa227cd043719c7eff9d2b7e4bf0/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:25:45.042242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:45.072814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0840-2717-4431-b3f9-b75003fc0653 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:45.113361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:45.114023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.114389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.114723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.133108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:45.138845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:45.139854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:45.140708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 313/610] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:45.146626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.290889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c6a44-948f-4c67-87ac-127285ec4045 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:45.306545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:45.338627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:45.339581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.340209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.340707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.341607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:45.342142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:45.344482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:45.354387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 298/611] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:45.360343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.485851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925a635-1e8d-4d21-b38f-41a38a75d483 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:45.521372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:45.522010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.522323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.522596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.531945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:45.534755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:45.536199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:45.550990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:45.552174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:45.553061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:45.553061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:45.553061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:45.553061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 314/612] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:45.558076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.707217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aeb793-cb26-43c4-97a9-4310ea8036fc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:45.756737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:45.757980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.758292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.758566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.759234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:45.759540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:45.760959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9034f-58d2-4ee3-a0ca-2a9685f12255 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:45.775810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 299/613] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:45.781363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.871329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9a62-061c-4244-a8b5-149a1d2a2f99 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.871855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9a62-061c-4244-a8b5-149a1d2a2f99 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.872134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9a62-061c-4244-a8b5-149a1d2a2f99 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.873780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 300/614] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:45.878725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.879006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.879281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.900611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3daf63-8e36-498a-97a5-8f5ce509b1a0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:45.919033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:45.920266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:45.936226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:45.936858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:45.937136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:45.937411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:45.945442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:45.947147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:45.958709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:45.959638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:45.961109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 315/615] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:45.966260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:45.979128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.023574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.075712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b58b43-4cbb-4376-9a48-47882c1cd1c5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.102235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 301/616] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:46.125600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.126575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.127057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.127527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.128532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:46.129035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:46.130684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:46.157515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 316/617] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:25:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:46.162958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.312856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ff5d8-e4c9-4c3d-9365-5ed6a717c999 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.362575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.363218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.363497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.363770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.380966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:46.382347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:46.401144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:46.402287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:46.403229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:46.403229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:46.403229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:46.409816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 302/618] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:46.416109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.451705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.474204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.588659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164d75-e844-48e2-98e3-6e58aa03ce98 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.598910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3d01b-eef7-4e5f-935d-e39e06da9c90 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.611334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.636543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.637704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.638067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.638510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.639210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:46.639482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:46.640850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:46.653094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 317/619] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:46.658309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.683318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.683960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.684242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.684510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.722989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:46.728893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:46.729822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:46.730710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:46.738537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 303/620] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:46.744531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.762923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2339a6b-d2a1-4261-a13b-8bff10710ff6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.800458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.801087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.801368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.801645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.809149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:46.810642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:46.823708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:46.824634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:46.825539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 318/621] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:46.830278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.877352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.900898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:46.925431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13623ea4-283e-4c54-8c8f-6fc73a7330c7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:46.969260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:46.970154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:46.970587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:46.971021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:46.971702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:46.972007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:46.973403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:46.989944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 304/622] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:46.996196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:46.999263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9211-2411-4cbc-90ee-bfb959349980 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.074700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.075322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.075603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.075875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.076573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:47.076875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.078332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.105809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 319/623] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:47.112129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.173450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7df5-e6a2-44fd-8542-ae62335bef8d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.215236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.215868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.216143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.216420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.223619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:47.228073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.229541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.234000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a3e7ef95-19b3-48cd-acf1-43bc8f990392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:47.243897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:47.245187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:47.246115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:47.246115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:47.246115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:47.246115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 305/624] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:47.247743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ffd5a-6765-4b1d-bac8-95c3438376b0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.251293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd077e0-29e9-49e5-8f10-6eb49df9faf4 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.251539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd077e0-29e9-49e5-8f10-6eb49df9faf4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.251816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd077e0-29e9-49e5-8f10-6eb49df9faf4 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.257308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 306/625] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:47.257308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd077e0-29e9-49e5-8f10-6eb49df9faf4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.289846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.290993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.291484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.291989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/da9b976e48d3468b82d9168efced0b68/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.308993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=da9b976e48d3468b82d9168efced0b68, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.310765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2999c-9177-4c70-bf64-b8c11c88b8b4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.365862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd077e0-29e9-49e5-8f10-6eb49df9faf4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.396665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 320/626] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:25:47 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/da9b976e48d3468b82d9168efced0b68/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:25:47.399808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e16b2-60bc-4969-a692-59379e69896b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.400347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e16b2-60bc-4969-a692-59379e69896b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.401086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e16b2-60bc-4969-a692-59379e69896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.407043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.407715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.408050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.408322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.408988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:47.409294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.410800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.424907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 307/627] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:47.427174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4d5e16b2-60bc-4969-a692-59379e69896b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:47.428717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4d5e16b2-60bc-4969-a692-59379e69896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:47.432394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.520339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 321/628] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:47.531990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:47.617820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734903f-00e9-46c2-a99c-c789774391d6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.662886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.663534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.663814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.664247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.672370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.673827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.680110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f2e5-6215-47d6-91db-9e4f15b1788d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.680972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f2e5-6215-47d6-91db-9e4f15b1788d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.681439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f2e5-6215-47d6-91db-9e4f15b1788d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.683225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 322/629] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:47.689183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.689824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.690221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:47.691332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:47.692465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:47.693856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 308/630] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:47.698738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.715239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:47.722726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:47.809717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa31a5-4ff5-48da-9c03-8a15d75efd10 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:47.822515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:47.861764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:47.862395 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:47.862673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:47.862950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:47.863686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:47.863993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:47.870010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:47.873691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:47.883837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 323/631] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:47.888360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.913224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 309/632] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:47.918528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:47.997545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93c07-3003-4569-b68e-c4fcaa424eee None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.033689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.034397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.034679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.034954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.046451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-386d7da3-492e-4fba-aa49-2c2d1aa5527e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:48.051338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:48.052269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:48.053096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 324/633] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:48.054298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425680-d6dc-4530-bf00-6ced4201d6e4 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.054298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425680-d6dc-4530-bf00-6ced4201d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.054581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425680-d6dc-4530-bf00-6ced4201d6e4 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.056081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 325/634] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:48.058522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71425680-d6dc-4530-bf00-6ced4201d6e4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.083774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd51df-c560-42b7-802a-a74fb04f396b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.120154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.121215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.121699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.122231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.124352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.166872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.170381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71425680-d6dc-4530-bf00-6ced4201d6e4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.214878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.215477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.215645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.215765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.216009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.216667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:48.216966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.217119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.218428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f10ef-12a2-4faf-9d26-6377abd72504 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.231871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 326/635] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:48.233173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.233458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.233733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.252950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:48.254067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:48.323261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 310/636] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:25:48.324021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.329100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 327/637] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:48.329828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.404667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.409060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.453270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98f2f1-a51e-4404-9f83-47a169302cc1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.454995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021774-084b-4193-a29f-55eeaf0e66ab None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.493141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.493721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.494160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.494556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.499091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.500067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.500635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.501067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.506448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:48.512401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:48.513317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.514232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:48.514987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.514987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.521337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 328/638] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:48.522517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:48.531532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:48.532734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:48.533639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:48.533639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:48.533639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:48.534250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 311/639] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:48.535090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.636429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab926c-a7f3-4ba9-852e-4f5e4ab0ccaa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.650112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.659916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02484e-9ab1-460c-ae46-3013336a3239 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.671285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.671941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.672220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.672471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.680131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.682261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.683702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.696157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.696763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.697041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.697312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.698264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:48.698567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:48.699835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:48.700725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:48.702316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:48.702316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:48.702316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:48.702499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.702571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 329/640] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:48.702571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.718220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 312/641] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:48.718220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.824571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e31d26-65c1-4c7b-b5b2-d535939d16f7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.824747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a784c-2704-474b-b454-aa1367fb3331 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.859745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.859948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:48.860375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.860492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:48.860650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.860748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:48.860916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.861014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:48.861577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:48.861858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:48.863454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:48.871680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:48.877677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:48.878913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:48.879777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 313/642] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:48.880497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.880568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 330/643] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:48.882157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:48.910036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.924450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:48.982980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bc6d4-3504-4f36-a0a3-3bc56b25d525 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:48.992749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a4f2e-cc0a-469c-b0e8-45cc87840787 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.018250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.018845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.019126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.019393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.020026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.020326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.021688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.028202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.028820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.029099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.029371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.039530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 314/644] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.039530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.043989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.045419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:49.061788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:49.063002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:49.064102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:49.064102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:49.064102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:49.064102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 331/645] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:49.064102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.145375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.145881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2b52-4acc-4515-b153-fd08f6f33226 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.148128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.167820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02e57-ec8c-4d6a-8520-f731bb428904 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.188220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.189123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.189399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.189820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.199127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.200586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.212063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.212063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.212063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.212063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.212432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.212432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.212432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:49.212663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:49.214096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 315/646] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:49.215364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.227188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 332/647] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.227188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.321334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e930390-23ed-4308-90e6-3c2ac299b1c6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.329483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd3d78-f034-4b51-bbde-5109769864ff None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.357186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.358185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.358706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.359135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.363186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.363798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.364081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.364356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.365000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.365307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.366733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.371861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.373295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.373809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.386044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 333/648] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.386044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:49.397914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:49.399137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:49.400124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 316/649] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:49.400755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.432650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.491856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a104-1fb5-4113-a9d6-503d80b106a1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.509397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5d72d-759f-42ce-853a-743976b2070e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.530790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.531411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.531709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.532426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.533174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.533481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.534945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.547809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.548651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.549098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.549512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.553639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 334/650] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.554503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.562759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.564407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:49.579477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:49.580613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:49.581719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:49.581719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:49.581719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:49.583135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 317/651] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:49.584246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.618523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.669400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf2f9-137d-4569-ba01-8cc758fb7201 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.673169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.700424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a02624-0722-460f-be7d-55a35ad131a5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.709087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.709725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.710000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.710303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.718804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.720236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:49.732083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:49.733230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:49.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 335/652] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:49.734907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.740809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.741420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.741699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.741970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.743214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.743518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.745118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.757801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 318/653] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.758896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.844889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602346-7707-427d-88bf-a35c46cf5c6c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.856965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afcdb4-095f-4e6b-87f6-f69f0af2a94d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:49.860717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.878178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.878776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.879053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.879326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.890116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:49.890720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:49.890996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:49.891267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:49.891899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:49.892200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.893869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:49.895312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:49.906298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 319/654] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:49.907192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:49.954933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:49.956273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.008602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3177-7b75-4e7c-ad9c-154ecdb2398a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.041798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.042419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.042693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.042963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.043676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:50.043979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.045411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf9bd-a2c9-428e-8a18-2b8432c181fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.051883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 336/655] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:50.052562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.066277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 320/656] 10.222.0.60 () {56 vars in 1139 bytes} [Fri Sep 9 11:25:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:50.067535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee7bc-efb1-4b96-869e-6106ef3425e2 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.067821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee7bc-efb1-4b96-869e-6106ef3425e2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.068089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee7bc-efb1-4b96-869e-6106ef3425e2 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.069499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 321/657] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:50.070175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfee7bc-efb1-4b96-869e-6106ef3425e2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.073684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.123527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.153545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3133321-9c12-41d1-a593-e8f94b218156 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.169837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfee7bc-efb1-4b96-869e-6106ef3425e2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.188218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.188814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.189106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.189373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.190121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=10d651db5bf74166bff093d317d83a6c) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:50.190434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.191842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.198529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 337/658] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/role_assignments?user.id=10d651db5bf74166bff093d317d83a6c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:50.199303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.203690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.204328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.204602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.204868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.214740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.216157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:50.228925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:50.230398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:50.231565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:50.234997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 322/659] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:50.235978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.290598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.313132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138da36-b345-4e2b-a6fa-1c91c25660f5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.341971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88eb46-4129-4d01-a7b6-467bfaf4f190 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.354500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.355553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.355699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.355799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.356231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.371261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.372712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.387006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.387798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.388077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.388347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e1390c06-6cef-43e0-b022-59b0257f9486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:50.392924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:50.393923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 338/660] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.400212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.416461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:50.417987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:50.471970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.473353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.497088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 339/661] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:50.497941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.570908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.584037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 323/662] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:50.585168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.612454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.621756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf02f2-7b78-41ef-a65f-7b5f80b82baf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.659047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.659653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.659946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.660212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.660914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:50.661214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.662852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.688297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 340/663] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:50.689569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.737058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12577e-a1a9-4338-864f-e8a10593f2ac None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.791636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.792392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.792766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.793179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.793916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:50.794341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.795801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.803881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b2224-af11-4333-b285-2585d4c58cc8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.808384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 324/664] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:50.809214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.830430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.845888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.846697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.847076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.847510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.858400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:50.859898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:50.874711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:50.875843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:50.876956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:50.876956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:50.876956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:50.880240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 341/665] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:50.881055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:50.885096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:50.915795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bee88-99e4-4f57-b77d-20ce6e1ea0f1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:50.950428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:50.951034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:50.951313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:50.951587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:50.999433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305a5cb-4820-4b1d-9841-67455843ac1b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.022201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.023360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.036907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.037538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.037816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.038115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.050345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:51.058694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:51.059818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:51.060695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:51.062222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 342/666] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:51.063209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.097340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.098896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.128017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 325/667] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:51.128858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.185296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71234f38-e6de-4f31-aecd-c6b36df23537 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.220808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.221462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.221743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.222018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.233956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.234163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.243738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d396-a95f-4378-8d0f-84197c60ad0c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:51.247515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:51.248562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:51.249709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:51.251389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 343/668] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:51.252289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.284233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.284967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.285347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.285735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.286500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.286903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.288370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.304247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 326/669] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:51.304247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.336224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.353688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.363108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207e231-c6d9-417a-b654-211765eead80 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.397820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.398682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.399536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.399806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.400464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.400759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.402138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.409858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 344/670] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:51.409858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.414778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4303-d3fd-4f63-acb1-be5253b9124a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.449702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.450328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.450599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.450863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.462004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.463485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:51.478074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:51.479680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:51.480826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:51.480826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:51.480826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:51.480826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 327/671] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:51.480826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.523524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b4ef-22fe-47e7-adab-f4f22c6a6046 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.559886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.560494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.560777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.561047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.561734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.562039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.563485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.574989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.578075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 345/672] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:51.578993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.595107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.596195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b25b9-1d3f-4ce6-88da-46011b7d3186 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.629676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.630338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.630625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.630896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.638235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.639732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:51.651595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:51.652512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 328/673] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:51.653430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.691957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468ce57-fcf1-4079-b997-be5d3e78b6fb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.731189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.731811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.732093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.732365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.743579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.745128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.758818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00b48-4253-4b23-ba56-8d2629a39cd5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:51.761285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:51.762418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:51.763566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:51.763566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:51.763566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:51.763566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 346/674] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:51.763566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.798560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.799426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.799901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.800351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.801183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.801668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.803139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.815784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 329/675] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:51.816641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.822836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.838898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:51.874535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa038-0bad-4a84-a98a-4f783e1e2b6e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.909459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.910070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.910372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.910650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.911373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=2eccd6d01b9a48cf946ea37543cf8c3a) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.911683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.913119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:51.919219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 347/676] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/role_assignments?user.id=2eccd6d01b9a48cf946ea37543cf8c3a&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:51.920007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:51.935403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418a01-126b-4923-ac59-037a74f373d7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:51.986237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:51.987537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:51.988000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:51.988731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:51.990256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:51.990916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:51.993043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.010277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 330/677] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:52.011626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.068645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef88a69-2f5e-4773-8bf2-37f6e818ecb1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.102798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.108107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.118518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.119135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.119413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.119686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.120350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:52.120640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.121994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.135548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 348/678] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:52.136306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.158815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c34681-8a1d-427f-bc3c-16e6bda9b503 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.207960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.208599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.208877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.209148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.216625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.218091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:52.234301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:52.235242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:52.236171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 331/679] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:52.236673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.271034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877313a-3829-4b7d-a0b8-2adb98a20eb5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.314415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.315374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.315853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.316343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.317261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=b062fa227cd043719c7eff9d2b7e4bf0) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:52.317784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.319626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.334090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 349/680] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/role_assignments?user.id=b062fa227cd043719c7eff9d2b7e4bf0&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:52.335548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.357722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a848fb-3c3b-4294-8ed6-de41db79cda6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.410488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.444215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.447687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.448581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.448857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.449125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.472962 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.474518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c33dd730-d630-4b48-ab88-c7415e8ad72e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:52.494585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:52.496175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:52.497157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:52.500799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 332/681] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:52.501716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b893a29-88fd-455d-9558-f01d0e9f1a31 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.502648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ab4a-c4a9-4d3d-b865-289f76b3a000 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.503151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ab4a-c4a9-4d3d-b865-289f76b3a000 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.503798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ab4a-c4a9-4d3d-b865-289f76b3a000 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.505882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 333/682] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:52.510651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ab4a-c4a9-4d3d-b865-289f76b3a000 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.552524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.553610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.554149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.554659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.556276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:52.556820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.558636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a675996-ec88-4773-95c0-f8dc9f5857f8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.572153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 350/683] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:52.573554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.573895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.574119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.594262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:52.595922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:52.644110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ab4a-c4a9-4d3d-b865-289f76b3a000 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.677846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 351/684] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:52.678763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.700138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577315-7947-491e-935a-95c749386811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.700822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577315-7947-491e-935a-95c749386811 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.701105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577315-7947-491e-935a-95c749386811 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.701374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577315-7947-491e-935a-95c749386811 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.702938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67577315-7947-491e-935a-95c749386811 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:52.703247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67577315-7947-491e-935a-95c749386811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.706444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67577315-7947-491e-935a-95c749386811 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.738633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 334/685] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:52.747340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577315-7947-491e-935a-95c749386811 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:52.767345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.826661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:52.881978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577315-7947-491e-935a-95c749386811 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.896172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cf40a-1fbf-439c-9546-20ede90c4efb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:52.923447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.924313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.924590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.924861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.969496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:52.970123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:52.970402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:52.970672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:52.972730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:52.973043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:52.976050 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:52.988235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 352/686] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:53.009776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.087376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.089234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.132190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e3ded-df5c-47b0-896c-be8e98a17959 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.143239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:53.174509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:53.195444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.195773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.196549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.197332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.213451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.215320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:53.237807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:53.238855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:53.240372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:53.240372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:53.240372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:53.240372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 353/687] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:53.245521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.260560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 335/688] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:53.268282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.370981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714aa1-e881-4792-9f38-4cc59c8c29e0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.383254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7424351-a95f-4f09-a7a0-058dd79d610e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.418536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.419534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.420121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.420669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.421711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:53.422384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.423524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.424239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.424511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.424632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.424913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.439531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.441172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.441581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 354/689] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:53.447237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.447674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:53.464479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:53.465479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:53.466422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:53.466422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:53.466422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:53.466422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 336/690] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:53.471747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.474139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:53.578490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da71ea-3346-4cfe-8974-ce9b2e8221dd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.587624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a3441-9738-409c-b98e-d33761bc2d0e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.624285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.624812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.625088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.625360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.625879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.626124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.626923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.627404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.627659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.628061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.628894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:53.629399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.631068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.655519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 337/691] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:53.662191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.724123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 355/692] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:25:53 2022] POST /identity/v3/users => generated 254 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:25:53.728786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:53.798717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df503-bb47-4134-8093-e35dffa9d610 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:53.849978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:53.850903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:53.851327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:53.851851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:53.864240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:53.866007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:53.879082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:53.880231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:53.881333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:53.887394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 338/693] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:53.893417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:53.991757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:54.016234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b11da0-0c8a-4da7-8341-8bdc0de5f1a9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:54.052091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:54.052697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:54.052968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:54.053233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:54.053992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:54.054383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:54.055858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50ff60-77a6-47d9-915d-b83b77a05b7e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:54.068231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 356/694] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:54.074579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:54.199875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb145b-e9d0-424b-bd4e-91719343691f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:54.217596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:54.239376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:54.239977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:54.240250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:54.240522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:54.241248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=da9b976e48d3468b82d9168efced0b68) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:54.241555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:54.243589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba913-1ea1-444f-ba84-7f7d633771af None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:54.255791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 339/695] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:54 2022] GET /identity/v3/role_assignments?user.id=da9b976e48d3468b82d9168efced0b68&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:54.390159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63095-a803-445f-8594-b70c5403ae01 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:54.390474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63095-a803-445f-8594-b70c5403ae01 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:54.390706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63095-a803-445f-8594-b70c5403ae01 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:54.392345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 357/696] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:54.397865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:54.398199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:54.398447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:54.425289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:54.426624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:54.506099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 340/697] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:54.843273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d63095-a803-445f-8594-b70c5403ae01 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:54.960953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d63095-a803-445f-8594-b70c5403ae01 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:55.000262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:55.001688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.002228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.002697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.003478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:55.005028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:55.008978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0804bf57-08fa-46e4-81a3-f093d90bc8e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:55.017065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:55.018119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:25:55.018982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:55.021426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 358/698] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Sep 9 11:25:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:55.027488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:55.059286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0ac4-7dc9-4ac4-953f-5f7bfd46f7dc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.059572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0ac4-7dc9-4ac4-953f-5f7bfd46f7dc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.059851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0ac4-7dc9-4ac4-953f-5f7bfd46f7dc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.061442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 359/699] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:55.066583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.066871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.067155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.088458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:55.090383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:55.155718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f5e7c-bc63-4f41-8156-628ba1aca329 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:55.166596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:55.216171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:55.216773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.217052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.217324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.217958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:55.218289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:55.219652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:55.228717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 360/700] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:55.236239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 341/701] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:25:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:55.241685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:55.361965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d4151-a3c8-46f3-9923-8d4a2bb59e3c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:55.393752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:55.399905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:55.400519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.400802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.401081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.401723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:55.402029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:55.403419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:55.409692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 361/702] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:25:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:55.598918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:55.718971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3326e7-8d8f-4aac-bc58-53f2129bfe35 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:55.751099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:55.762532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.762532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.762532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] PATH_INFO: `/v3/services/network` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.762532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:55.762532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f16db2d1-2470-4019-846a-c3e3225bcf79 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:55.766660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:55.767630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:55.768847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 342/703] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:55 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:55.775692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:55.860206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406255-387e-4b94-91a9-f27831b4b209 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.860537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406255-387e-4b94-91a9-f27831b4b209 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.860763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406255-387e-4b94-91a9-f27831b4b209 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.862151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 343/704] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:25:55.872100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:55.872100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:55.872100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:55.888068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:55.889550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:55.919578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:55.925623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c9e72-9aaf-4287-9385-b5a6dfe5d245 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:55.989784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 344/705] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:25:56.015347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:56.016032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.016252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.016522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.017170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] RBAC: Adding query filter params (name=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:56.017471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:56.018867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:56.030172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 362/706] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:25:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:56.035511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:56.166265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4faaf-03a1-4ec0-8c7d-ae3fdb73595d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:56.204260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:56.205253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.205730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.206228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.207119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] RBAC: Adding query filter params (type=network) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:56.207615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:56.209202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:56.216138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 345/707] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:25:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:56.366977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:56.439250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:56.496884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5007aa-d69f-4022-bc4e-bc1300836d35 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:56.531444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:56.532075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.532354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.532628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.543163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:56.548518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:56.549446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:56.550633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 363/708] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:56.557939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:56.606897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:56.690382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b360a6e-0026-415c-849d-60e14384040c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:56.740234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:56.740858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.741136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.741405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.742101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:56.742408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:56.743994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fccb7-eb96-4ec2-a536-e8cb258976ca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:56.757502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 346/709] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:56.764157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:56.864781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb475e-d88d-434a-aa9f-cc1cb55bc143 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.865103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb475e-d88d-434a-aa9f-cc1cb55bc143 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.865334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb475e-d88d-434a-aa9f-cc1cb55bc143 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.866980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 347/710] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:56.872012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.872295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.872564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:56.891404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:56.892730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:56.925905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa706b-a330-4705-afc0-153a360ee147 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:56.958136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:56.989026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:56.989675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:56.989942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:56.990243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.004884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 348/711] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:57.008574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:57.010680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:57.033107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:57.034851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:57.035732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:57.035732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:57.035732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:57.035732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 364/712] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:57.042403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.077043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:57.175969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86889a13-60b8-4c06-b989-20133f60c439 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.212978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.213530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.213809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.214097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.214847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:57.215158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:57.216587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:57.230821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 349/713] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:25:57.239266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.319054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.356786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90d181-f7d6-48ec-b80b-4f89bc5c651f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.412123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.413177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.413711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.414217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.423899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:57.425669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:57.438286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-101e6698-620b-4f1d-9b8d-5ad86691c1e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:57.440431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:57.441400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:25:57.442767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:57.446131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 365/714] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:57.448665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b897-9551-46de-8c18-c5a3d67b0c61 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.449446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b897-9551-46de-8c18-c5a3d67b0c61 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.449914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275b897-9551-46de-8c18-c5a3d67b0c61 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.451590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 366/715] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:57.456455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b897-9551-46de-8c18-c5a3d67b0c61 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.467481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa214d-36b0-43b2-a63a-41abe3d2a95f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.494163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:57.502910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.503538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.503800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.506170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.515106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-71a1f468-c5e3-48ff-8032-951d0acffefd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:57.520403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:57.521379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:25:57.522284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:57.526138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 350/716] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:57.527918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.528198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.528471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.549084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:57.550406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:57.582565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275b897-9551-46de-8c18-c5a3d67b0c61 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.626102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 351/717] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:57.626102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.627341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.628167 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.628617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.629038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.630012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:57.630485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:57.637690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:57.671739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 367/718] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:57.677334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.754515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:57.778350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793e15-c6d0-4800-8262-75f090ced4f9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.859422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.860273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.860737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.861144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:57.862032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:57.862498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:57.866234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:57.883769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 352/719] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:57.887739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:57.891959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:57.914715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e99f-c593-46e6-922a-ff1cfebca11f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:57.960433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:57.961041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:57.961317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:57.961608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.029709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0bd0-d5fb-402b-a17c-7206902e40f7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.046692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.049879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.069389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.070096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.070464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.070844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.073594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.083718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.085281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:58.100508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:58.101724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:58.102913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:58.102913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.102913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:58.103861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 353/720] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:25:58.104948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.166872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.182562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 368/721] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:25:58.183705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.237244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc2ab-dbfd-4747-8ea1-debf980d615e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.280040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.280774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.281355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.281473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.293595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:25:58.298959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:58.300067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:58.301061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 354/722] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:58.301748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.315559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95ead-56ed-4e3c-8581-d20ed9197cef None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.354094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.357435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.358235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.358653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.359095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.360290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:58.360743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.362630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e61-99d9-4db7-b390-ccaaed5adef5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.374644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 369/723] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:58.375759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f99f8-3c83-4b49-96b0-dd439f97db84 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.381657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f99f8-3c83-4b49-96b0-dd439f97db84 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.381657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f99f8-3c83-4b49-96b0-dd439f97db84 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.381657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 370/724] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:58.381657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f99f8-3c83-4b49-96b0-dd439f97db84 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.419142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aacc6-d096-4738-a8be-a0cb5d328818 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.433239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.453230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.454096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.454514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.454962 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.466365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.468391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:58.483926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:58.484918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:58.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:58.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:58.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 355/725] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:58.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.493053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f99f8-3c83-4b49-96b0-dd439f97db84 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.531810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.532609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.532890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.533161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.533823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:58.534141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.535487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330c5a-aaf3-4bc9-b9dd-1839bd9ade25 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.545343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 371/726] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:58.546609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.546892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.547159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.568013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:58.569230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:58.595297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.599943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca879-91bc-4fa9-b3ae-e61bd8074700 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.639951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.640581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.640866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.641135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.654402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 372/727] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:58.654402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.654623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.654623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:58.662313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:58.663281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 356/728] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:58.664216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.713341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.773299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ea470-8633-4b78-977b-48e25fe1cb7d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.779776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce6c85-7436-4c32-9a77-e1fc4dce8386 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.810564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.811119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.811401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.811676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.815460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.816062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.816338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.816611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.817249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:58.817552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.818952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.822822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.824264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.834560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 373/729] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:58.834560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f1ec0dde-77d5-481b-9fd9-88e14771da4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:58.843941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:58.845083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:58.846249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:58.846249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:58.846249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:58.848157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 357/730] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:58.849612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fcd02-fbee-4032-a410-371c07766986 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.849895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fcd02-fbee-4032-a410-371c07766986 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.850417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fcd02-fbee-4032-a410-371c07766986 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.851515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 358/731] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:25:58.852432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fcd02-fbee-4032-a410-371c07766986 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:58.865128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.948371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:58.951814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a3c28-0ded-4759-bc79-7b977f189a34 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.959573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fcd02-fbee-4032-a410-371c07766986 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:58.987217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.987851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.988128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.988402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:58.989146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:58.989453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:58.990907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:58.992564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:58.993173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:58.993473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:58.993743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.000345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.001765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.008080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 374/732] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:59.008874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6d0e5ac9-532b-4776-b716-8ea3a63e55d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:25:59.013946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:59.015013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:25:59.015917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 359/733] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:59.016405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.016611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.016884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.034592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:25:59.035781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:25:59.100244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.104915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 360/734] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:25:59.105692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.113369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba8372-e460-44b1-8b93-c996274bc022 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.151420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.152036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.152311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.152584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.153225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:59.153529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.154921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.172388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 375/735] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:59.173246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.195351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.226301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c5880-6851-422b-8a55-f8aac6f646b1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.261297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.261909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.262211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.262482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.263129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.264505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.287574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da94f8-43f5-49c6-ac29-6e35c67b35d6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.321373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.321986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.322293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.322565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.371152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 361/736] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:25:59 2022] POST /identity/v3/services => generated 251 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:59.372057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.404000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.405106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.425878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.446570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.500159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcaa5b-f711-47b0-8ac9-bddfe3435a8f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.520597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 376/737] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:25:59.521475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.545801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.546455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.546731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.546998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.547698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:59.548002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.550088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.563054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 362/738] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:25:59.563573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.642873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb35382-3ce4-451c-a7c9-294057e95853 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.662163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.669204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4193652-86a4-403d-9546-8faf7f97c921 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.679314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.680107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.680379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.680656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.685986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.689843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.691612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.704251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:25:59.704408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:59.705304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:25:59.711298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:59.711864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.711864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.711864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.712071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 377/739] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:59.712148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.712217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.712285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.790388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 363/740] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:25:59 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:25:59.791248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.831417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e013ec-7643-4dc7-8155-21f4d8bbb0a9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.893189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.895029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.895459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.895871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.909922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.911686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:25:59.934370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:25:59.935509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:25:59.936396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:25:59.936396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:25:59.936396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:25:59.936396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 378/741] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:25:59.937278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:25:59.942660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a1a38-3120-40c3-aa3e-830ceec5ab1b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:25:59.977994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:25:59.992394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:25:59.994230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:25:59.994230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:25:59.994230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:25:59.994510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:25:59.994779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:25:59.997414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.004410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 364/742] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:00.006007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.067502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.070242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87241314-22f6-4fbb-9461-9b51b40df614 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.119123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.120162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.120445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.120721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.121807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:00.122311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.123997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.147617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 379/743] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:25:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:00.153498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe7f50-89e5-45fe-8626-71927e13a939 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.155084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.233421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.234047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.234850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.235128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.235786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:00.236089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.237757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.261383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 365/744] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:00.266807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.267976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.280323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a395b1-f4e3-428e-85c4-c32da816b4ee None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.329024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.330046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.331076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.331962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.413902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.415500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.500393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.504166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.508972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865c377-969f-42b3-b5e9-f3d4cfa2a4e6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.523262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 380/745] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:00.529798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.574420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.575310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.575585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.575850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.590707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.592193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:00.612726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:00.613674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:00.614613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 366/746] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:00.620509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.700633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114c414-e801-480d-aad5-207a8ad28689 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.735194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.735824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.736104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.736374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.747214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.748656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:00.764282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:00.765271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:00.766219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:00.766219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:00.766219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:00.766219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 381/747] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:00.775046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.818476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.828010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5222551-9fd8-44a7-aeeb-ec50483af166 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.898837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.899821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.900849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.901307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.902217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:00.902714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.904376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.916949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 367/748] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:00.922101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8e4f-1156-437b-953a-0c99c3561c8d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:00.923215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:00.954908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:00.960139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:00.966365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:00.971185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 382/749] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:00.979063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:01.099174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a9c6-b9fe-425d-9cb8-2a4382a25ad9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:01.114415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:01.136311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:01.136951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.137203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.137477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.144486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:01.145910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:01.158110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:01.159048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:01.159933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 383/750] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:01.166643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:01.216166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76680cff-fa5f-4887-b6b3-1a7c4ede1239 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:01.276260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a969-7f37-4c90-8a02-a6b7b717af07 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:01.309889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:01.310533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.310811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.311083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.311807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=295dd5a7b7774239bf95df04555611f1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:01.312113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:01.314341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:01.324311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:01.325492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.325970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.326196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 368/751] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:00 2022] GET /identity/v3/role_assignments?user.id=295dd5a7b7774239bf95df04555611f1&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:01.326844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.327816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:01.328325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:01.329982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eabf3-65e9-4bf5-a246-f78d491e9480 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:01.344693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 384/752] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:01.351499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:01.480203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:01.497898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9008-f641-4ce8-9ac5-74d2470adfca None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:01.535534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:01.536208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.536488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.536761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.537500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=da9b976e48d3468b82d9168efced0b68) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:01.537808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:01.539309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1468dc-9d73-4de3-9759-c842a7434e06 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:01.546771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 369/753] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:01 2022] GET /identity/v3/role_assignments?user.id=da9b976e48d3468b82d9168efced0b68&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:01.780675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3e656-0858-43c2-aa71-5702cecbe8a2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.780858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3e656-0858-43c2-aa71-5702cecbe8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.781145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3e656-0858-43c2-aa71-5702cecbe8a2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.782914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 385/754] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:01.788106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:01.788392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:01.788667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:01.813953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:01.815172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:01.896085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 370/755] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:02.318122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3e656-0858-43c2-aa71-5702cecbe8a2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:02.340605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:02.452087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3e656-0858-43c2-aa71-5702cecbe8a2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:02.500833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:02.501471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.501744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.502009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.502633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:02.503992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-92dcbe54-cb0a-4aa1-b171-5aff44178879 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:02.515090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:02.516150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:02.517005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 386/756] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Sep 9 11:26:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:02.522526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:02.602637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baf9-1b5d-4bfc-bfb6-44142d539ae7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.602923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baf9-1b5d-4bfc-bfb6-44142d539ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.603119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baf9-1b5d-4bfc-bfb6-44142d539ae7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.604872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 387/757] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:02.611151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.611454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.611683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.633318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:02.634551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:02.666495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:02.714251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6ff7c-082c-4e91-ab31-5494b0ccc299 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:02.746331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 388/758] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:02.796723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:02.797342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.797618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.797890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.798557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:02.798858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:02.800336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ee9f-0f2b-4efa-a39b-9422926b35b0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:02.809198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 371/759] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:26:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:02.814927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:02.826172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bfd-a324-4ac0-872f-20751bfa0c4c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.826746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bfd-a324-4ac0-872f-20751bfa0c4c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.827228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bfd-a324-4ac0-872f-20751bfa0c4c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.829010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 372/760] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:02.834032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.834563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.835039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.838172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:02.875066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:02.876618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:02.927655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9486df-77c3-481f-826d-e3610bcb061a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:02.966640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:02.968248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:02.968910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:02.970700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:02.971573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:02.972082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:02.973826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:02.986206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 389/761] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:26:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:02.992111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.014011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 373/762] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:03.070956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:03.096662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf4b64-923f-4f4a-b3bb-f5b78558b8c7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:03.107145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.132008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:03.132530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:03.132796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:03.133069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:03.133776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] RBAC: Adding query filter params (interface=public, service_id=e0889ab2dc1744288c74bf486fc21b36, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:03.134102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:03.137787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:03.149399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 390/763] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:02 2022] GET /identity/v3/endpoints?service_id=e0889ab2dc1744288c74bf486fc21b36&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:03.150713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.255517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b565363-4c47-44e7-8da5-4ba1d15d9bcd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:03.263034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:03.292770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:03.293516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:03.297270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:03.297270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:03.297270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:03.297270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:03.306190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 391/764] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/services => generated 772 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:03.334513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9f7e-bd94-4d35-9497-24d770d794b8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:03.419051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:03.420040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:03.420504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:03.420955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] PATH_INFO: `/v3/services/network` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:03.421737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:03.423373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:03.436235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:03.437296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:03.438176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:03.440780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 374/765] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/services/network => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:03.446535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.452991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:03.474866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.600223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c4f0e-9f56-4dd3-bf92-fb97c42973b2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:03.620801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28453571-54ef-4585-8e13-c88a29cc3171 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:03.640272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:03.641421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:03.641851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:03.642324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:03.643636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] RBAC: Adding query filter params (name=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:03.644245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:03.645778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:03.651182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 392/766] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:03.659014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.688701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:03.689256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:03.689527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:03.689795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:03.696374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:03.710935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:03.715887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:03.716852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:03.717871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 375/767] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:03.729902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:03.782119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:03.940514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b5600-3300-4986-8681-da0b8c789c33 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.001254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295a087-ab68-4a85-bfb6-c9c09edb270f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.007913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.008547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.008815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.009082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.009746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:04.010171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.011535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64f516-fb7d-4952-94e5-a68df1f398e5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.029547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 376/768] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:04.032346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703517f5-b4d2-4e28-8f9e-8b0b79cd75fc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.032627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703517f5-b4d2-4e28-8f9e-8b0b79cd75fc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.032900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703517f5-b4d2-4e28-8f9e-8b0b79cd75fc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.034817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 377/769] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:04.036417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703517f5-b4d2-4e28-8f9e-8b0b79cd75fc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.058745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.059409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.059820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.060159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.060880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:04.061180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.062627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415112dc-274b-49d0-935d-7faf0f956ae7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.068704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 393/770] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:04.073376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.073376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.073376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.090804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:04.092139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:04.134765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.178277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 394/771] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:04.179392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.188528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703517f5-b4d2-4e28-8f9e-8b0b79cd75fc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.224547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.225238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.225615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.225983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.237417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.239025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.255012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-7746af46-f495-42ff-a62a-af42e4276be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:04.255823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:04.256978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:04.258214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:04.258214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:04.258214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:04.264443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 378/772] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:04.264443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0eceb-3b24-4ed5-b38d-0cd0fbc21598 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.264443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0eceb-3b24-4ed5-b38d-0cd0fbc21598 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.264443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0eceb-3b24-4ed5-b38d-0cd0fbc21598 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.264443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 379/773] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:04.268026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0eceb-3b24-4ed5-b38d-0cd0fbc21598 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.308452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e0549-4afa-41b0-92d7-5371dd336bbc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.345249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.345975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.346356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.346720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.347524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] RBAC: Adding query filter params (interface=public, service_id=b35906e9867b41a380a181aacd6fc34b, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:04.347930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.349491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a259-60f9-4239-873d-de69ecb441ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.356523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 395/774] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/endpoints?service_id=b35906e9867b41a380a181aacd6fc34b&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:04.357407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.357786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.358187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.378512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:04.380066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:04.407710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0eceb-3b24-4ed5-b38d-0cd0fbc21598 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.421584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.452889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 396/775] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:04.453709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.454520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.454926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.455355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.456118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:04.456545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.459186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.459334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.475045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 380/776] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:04.480888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.532569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.585749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9db5c-4be5-4da2-b1bb-3c4012b46bfe None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.610443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e5ce-7ea7-41b1-b6b7-2fdeaccf016a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.623348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.624145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.624639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.625164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.625968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.629460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.635991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 397/777] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:04.637064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.652690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.653369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.653618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.653890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.665229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.667441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:04.681131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:04.682694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:04.683656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:04.686411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 381/778] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:04.691807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.754372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.768384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a600-9bbe-453f-97cb-4db3ebbc1cfe None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.802501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:04.825814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.827544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.827847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.828126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.844926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:04.851018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:04.851950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:04.854330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 398/779] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:04.855067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.867821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70dce3-7584-4fe0-93dd-597201449c89 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:04.910313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:04.910918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:04.911198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:04.911467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:04.912197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:04.912498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:04.913924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:04.931791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 382/780] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:04.933301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:04.982318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff3bc-b831-4cf1-9453-c287d6071451 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.022869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.023820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.024302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.024795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.030433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.034610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0f4e-7f22-400e-b55e-7fc96dc41e16 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.036823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:05.044182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.046405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:05.047254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.047907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 399/781] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.048852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.075595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.076209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.076486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.076757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.077421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:05.077723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.079118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.087705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 383/782] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:26:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:05.089087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.099939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.163612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916be72-9add-4d53-b12e-294abe46031e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.193219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958e2b-efe5-4975-b574-f39d0341d3a7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.200113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.201265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.201690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.202188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.212910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.214790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.228460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.229657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.229976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.230273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.230930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:05.231229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:05.231435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.232522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:05.233627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:05.233627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.233627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.233830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.235440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 400/783] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.236498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.250240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 384/784] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:05.251213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.279627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.332420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.342817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef67579-bd75-4203-b0ce-91480250b2ff None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.385124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.385693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21894f7-7b94-4905-b29a-ed2252c374c1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.385972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.386359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.386698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.397552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.399151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:05.416172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.417530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:05.418641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:05.418641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.418641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.423881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 401/785] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.423881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.534804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a4d19-f5b2-4fbd-944f-e0cc74fc1712 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.578975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.579897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.580252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.580706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.582273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.583240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.584016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.584470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.585286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:05.585762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.587350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.593183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.594726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.598403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.603215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 385/786] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:05.604484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:05.612350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.622652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:05.623763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:05.623763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:05.623763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.623763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 402/787] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.623763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.703173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.746155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a320f-1df1-40b4-80cf-0b252be754d3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.746388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dcd88-f920-4ac2-8369-40f4a2dd7cdc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.783488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.783903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.784283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.784508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.784610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.784781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.784925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.785098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.785680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:05.785983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.787364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.791823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.793368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.803397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 386/788] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:05.803397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-809f4730-4f0f-4578-82d6-5175d9c51b8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:05.809376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.810329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.811348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 403/789] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.811844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2457-dff9-4104-8698-7df2caf79526 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.812731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2457-dff9-4104-8698-7df2caf79526 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.812731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d2457-dff9-4104-8698-7df2caf79526 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.817921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 404/790] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:05.817921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d2457-dff9-4104-8698-7df2caf79526 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.877244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.913206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d3a59-f2b6-4ed5-a33e-7c9f70d7cb7d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.926691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d2457-dff9-4104-8698-7df2caf79526 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:05.944555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:05.951957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.953438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.953438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.953611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.954250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:05.954959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.956619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.963221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:05.964034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.964409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.964773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:05.974721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 387/791] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:05.974721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:05.974954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:05.974954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-51c2ebb4-25ee-4d73-98af-bd4932efdedb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:05.990369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:05.991354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:05.992309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:05.992854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 405/792] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:05.993564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:05.993888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:05.994212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.015920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:06.017131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:06.087017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 406/793] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:06.087786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.096900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf5a84-adc5-4173-a6d5-25cf51350ea5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.132972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.133748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.134161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.134532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.135410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:06.135950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.138029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.152547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.154309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 388/794] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:06.155341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.220735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.224729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831213e-b48d-478b-9f23-f9314e557fcc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.264193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.264985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.265371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.265735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.273168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.274746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:06.287045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:06.288222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:26:06.289400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:06.291182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 407/795] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:06.291596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7be7c-c4c9-4257-9adf-0969608e2260 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.292263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.339206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.340083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.340903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.341473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.342484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:06.342940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.345496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.369190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 389/796] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:06.370945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.407493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6ec4d-ba13-4255-a3bf-f59d2ae052b2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.409796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.442453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.443107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.443380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.443645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.445519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=e7b89441837347cba3200f9104c0f16c, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.447070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.482285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baef159-179d-40ac-8b2e-ea6bb8258924 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.492833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.515329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.515930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.516201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.516466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.517177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:06.517478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.518930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.535328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 390/797] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:06.536125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.556818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 408/798] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:06 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:06.567409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.661064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e81b6e-f1f3-4d43-8b5c-b6ee35acac87 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.701401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.702344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.702770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.703217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.715279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.717000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.723910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab888d51-95b9-468e-9852-9698c6e7a625 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:06.736384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:06.740883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:06.741780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:06.741780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:06.741780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:06.741780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 391/799] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:06.741780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.771605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.775868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.776565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.776844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.777115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.777709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.779569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:06.788433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:06.789619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:06.790677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 409/800] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:06.791470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.827892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:06.864593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f1af-7a5b-4a9f-97ee-62ec33559a17 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.906500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.907568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.908029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.908500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.920159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.922153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.924770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee87c5-fd5d-4234-89f6-690e27f262e2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:06.938676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:06.939692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:06.940593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:06.940593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:06.940593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:06.946173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 392/801] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:06.947410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:06.964674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:06.965311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:06.965590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:06.965855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:06.966555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:06.966989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:06.968859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:06.980806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 410/802] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:06.981449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.029451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.065617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e650f-e434-4bd5-8863-4149e5d78b47 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.078143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.091811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91849-577a-4ad2-8359-b6da17c2677d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.103038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.103645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.103924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.104244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.105017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:07.105315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.106734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79102b-a493-4769-9739-a8a2c0bcae39 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.112193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 393/803] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:07.113473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fb0e-1f2d-4a3c-84a8-c6ff95fe86be None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.113758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fb0e-1f2d-4a3c-84a8-c6ff95fe86be None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.114029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fb0e-1f2d-4a3c-84a8-c6ff95fe86be None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.118179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 394/804] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:07.119350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972fb0e-1f2d-4a3c-84a8-c6ff95fe86be None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.134296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.134911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.135240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.136106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.136752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:07.137056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.138710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.156700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 411/805] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:07.156700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.230709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972fb0e-1f2d-4a3c-84a8-c6ff95fe86be None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.265376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.266004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.266314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.266587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.268357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d26ad-b973-4108-bec2-ab58bb4ff7bf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.270024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.275334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.276814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-14e3eceb-3136-4f18-ac9b-3237ee915c08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:07.288178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:07.290706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 395/806] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.292186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.292801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.309113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:07.310381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:07.311119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.311907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.312540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.313842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.314664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:07.315228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.316743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.324450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 412/807] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:07.325335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.343396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.405184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 396/808] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:07.406287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.438538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f852c-79f1-4d03-8d1c-19930b91e9b5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.477033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.478103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.479574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.480144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.492171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.493953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:07.511474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:07.512408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:07.513397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:07.514515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 413/809] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:07.515051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.549187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6192413-097a-4a78-aa8f-eaed9604b462 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.575186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.584890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.585496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.586483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.586483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.586770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:07.587082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.588800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.606757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 397/810] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:07.606757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.614754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.646822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161f83-caa0-4dbf-b10e-ffdd85de0f17 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.684535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.685141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.685419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.685687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.686424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.687926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.727269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2580788-a0b9-4f00-a8db-008d6e43d381 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.759843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.760379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.760663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.760927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.761903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:07.762229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.763638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.775476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 398/811] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:07.776756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.802091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 414/812] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:07.802091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:07.832479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.910183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e313a94-2c1b-415f-9be1-cca61ba162de None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.937999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437ad89-bf77-4010-b6de-7c3efafdff9e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:07.957532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.958558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.958835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.959109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/295dd5a7b7774239bf95df04555611f1/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.965096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:07.978994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=295dd5a7b7774239bf95df04555611f1, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.980029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:07.980848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:07.981179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:07.981298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.981456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] PATH_INFO: `/v3/services/network` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:07.982272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:07.983631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:07.993641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:07.995003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:07.996251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 415/813] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:07 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:07.997072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:08.078951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 399/814] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:07 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/295dd5a7b7774239bf95df04555611f1/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:08.079895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:08.103877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63997c3-d6cf-4316-8ba9-47020c2d764e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:08.144540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:08.145206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.145483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.145825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/da9b976e48d3468b82d9168efced0b68/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.147846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=da9b976e48d3468b82d9168efced0b68, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:08.149425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:08.206131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:08.212957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3296a8-9b41-47cf-9540-f15e8cf729df None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:08.241096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 416/815] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:07 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/da9b976e48d3468b82d9168efced0b68/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:08.252139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:08.252780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.253058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.253446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.254220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] RBAC: Adding query filter params (name=network) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:08.254527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:08.255985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:08.262044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 400/816] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:08.267471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:08.334385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:08.393413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fa79-58e6-44bb-8d54-e5c591daab8d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:08.435778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:08.436404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.436684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.436957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.437605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] RBAC: Adding query filter params (type=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:08.437909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:08.439413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf906860-cd01-4439-b2c0-d963c79a1acb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:08.447524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 417/817] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:08.453581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:08.566294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6da65-488f-42d0-bf7c-91596a34e73a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:08.601343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:08.601883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.602191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.602467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.603150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:08.604593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11e9a4-8fd3-410c-84d0-cc5006041b7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:08.683812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 401/818] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:08.910904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ab1f1-1a01-412b-9ead-3bda9f1fe7dc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.911438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ab1f1-1a01-412b-9ead-3bda9f1fe7dc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.911438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ab1f1-1a01-412b-9ead-3bda9f1fe7dc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.913043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 418/819] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:08.919134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:08.919591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:08.919669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:08.938149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:08.939334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:08.951227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:09.018288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 402/820] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:09.297544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ab1f1-1a01-412b-9ead-3bda9f1fe7dc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:09.402358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ab1f1-1a01-412b-9ead-3bda9f1fe7dc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:09.415833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:09.437121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:09.437741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:09.448908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:09.448908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:09.449569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:09.454601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:09.455724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:09.456684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 419/821] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:09.461973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:09.568682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd1742-75fa-413a-a9b5-e6b8c1d2d8a9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:09.608432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:09.609051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:09.609329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:09.609603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:09.610308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:09.610611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:09.611979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:09.620824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 403/822] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:09.626173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:09.727937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cb038-384f-403a-8077-cf647663cd54 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:09.763079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:09.763702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:09.763973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:09.764240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:09.774552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:09.775997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:09.790449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:09.791413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:09.792840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:09.792840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:09.792840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:09.792840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 420/823] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:09.799152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:09.803019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:09.901763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f804dd-ef44-41cc-b463-04a980f5bed7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:09.935546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:09.936305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:09.936515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:09.936785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:09.937427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:09.937722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:09.939180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:09.951371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 404/824] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:09.956959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:09.984473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.059168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14cc93-e186-4c3f-b646-79cfccf525a4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.094721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.095339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.095613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.095875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.103055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.104494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e83b7514-0ff1-4e7a-9054-9e65760b1adc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:10.116502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:10.117468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:10.118457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 421/825] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:26:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:10.123943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.195979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.235241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445498-eb67-4a50-805e-2139a9c1dc17 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.263384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395963ab-45f9-41b4-bdf0-0936650de8e5 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.263679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395963ab-45f9-41b4-bdf0-0936650de8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.263908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395963ab-45f9-41b4-bdf0-0936650de8e5 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.265601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 422/826] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:10.270548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422b8a-322c-4a23-953d-3d4f0a4274cd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.270826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422b8a-322c-4a23-953d-3d4f0a4274cd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.271098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422b8a-322c-4a23-953d-3d4f0a4274cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.271273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.271866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.272145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.272411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.273130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:10.273453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.275028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.289492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-00422b8a-322c-4a23-953d-3d4f0a4274cd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:10.290729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-00422b8a-322c-4a23-953d-3d4f0a4274cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:10.294442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 405/827] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:10.300060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.390079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 423/828] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:10.428366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95386c6d-e93d-404b-8b9f-685316be9642 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.454405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.469473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.470138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.470463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.470737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.540885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.542924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.574091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226e3c1-0396-4312-962c-a1b283cc2aac None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.574420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226e3c1-0396-4312-962c-a1b283cc2aac None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.574627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226e3c1-0396-4312-962c-a1b283cc2aac None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.575955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 424/829] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:10.581355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.581636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.581911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.599869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:10.601056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:10.610397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.644483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 406/830] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:10.650044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.678919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.681307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 425/831] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:10.688114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.759428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08405ab-c204-44a0-be06-ce7dc8cdfe06 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.795822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.796542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.797111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.797111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.807602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.809774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:10.824758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:10.825745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:10.826952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:10.826952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:10.826952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:10.827651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9ce0-95f8-4cdb-bb04-9d8d2eaa4726 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.830786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 407/832] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:10.832180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.872764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.873509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.874534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.874534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.874814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.876131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f06ef3be-a72a-4f91-ae17-08072ce6c6a1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:10.888592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:10.889561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:10.890450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 426/833] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:10.891281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86c18e-2a47-4444-8692-7c2e86004fcc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.891281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86c18e-2a47-4444-8692-7c2e86004fcc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.891281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86c18e-2a47-4444-8692-7c2e86004fcc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.896425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 427/834] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:10.896425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86c18e-2a47-4444-8692-7c2e86004fcc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:10.906361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.926285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:10.955479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da30e9-ee10-4c5e-8882-ee2ae418c848 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:10.988372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:10.988964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:10.989239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:10.989512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:10.990174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:10.990475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:10.991824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fed0c-7212-4a6d-9c8e-5fbe4ba12018 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:10.999452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86c18e-2a47-4444-8692-7c2e86004fcc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.003688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 408/835] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:11.004832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.005113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.005382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.022222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:11.023338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:11.033880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.034523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.034799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.035066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.035711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.036013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.037378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.041828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 428/836] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.043176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.088541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 409/837] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:11.089405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.120065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.148586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.153211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fe9af-6e67-42cd-8fcb-5a292b422fdb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.184605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9785ca-7df5-4c5e-a473-c14f87e9ff97 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.188116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.188735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.189007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.189347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.200735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:11.206716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:11.207686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 429/838] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:11.208584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.222441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.223061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.223342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.223612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.231152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.232496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:11.243457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:11.244340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 410/839] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:11.245300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.320592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c1176-32e5-46a5-98bc-6c1c42731c55 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.345653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a267-5f58-4938-847f-761a26656c53 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.351753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.357868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.358508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.358789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.359057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.362114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.362114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.362114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.367275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 430/840] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.367275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.382661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.395438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.396091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.396305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.396581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.397272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.397577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.399048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.409933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 411/841] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.411924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.483754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2be1-a187-454e-a2a7-f6922908c332 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.522560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e312eb-31b5-41a3-83d4-9ae15f329e7e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.524950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.525652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.526011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.526322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.527033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.527344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.528793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.541997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 431/842] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.542935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.572274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.573112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.573427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.573886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.585207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:11.590987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:11.592136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 412/843] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:11.593203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.608938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.643212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.659883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c05c2-5124-4779-90ba-a8b915bf4365 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.700579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.701234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.701827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.702146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.707353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32cc86-a45d-4d1e-8a67-488546bdebbe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.713304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.715847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:11.730869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:11.731928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:11.732927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:11.732927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:11.732927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:11.732927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 432/844] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:11.733183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.749735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.750380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.750655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.750951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.751648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.751955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.753379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.770280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 413/845] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.774765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.851815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4da38-97bd-418d-9353-8292a2815b12 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.864640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.891756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a874a65-a5b6-4f40-a6e8-8af8d43a2c0d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:11.896382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.896827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.897105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.897372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.898069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.898414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.899903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.907410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:11.911533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 433/846] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.919165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:11.936833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:11.937844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:11.938440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:11.938932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:11.939838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:11.940490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:11.942138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:11.955614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 414/847] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:11.961810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.036283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa4ff5-942d-4c43-9088-881485994ca3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.082374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.082892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699d9e4-5c03-41ea-96dd-8b1c8087f72a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.083168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.083521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.083888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.094929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.096466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:12.113574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:12.114596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:12.115665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:12.115665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:12.115665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:12.115665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 434/848] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:12.122127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.129490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.130450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.130946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.131431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.139514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.141473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.153609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:12.154373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:12.155346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:12.156307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:12.158958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 415/849] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:26:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:12.166846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.209567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.254772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef295d4-b8c1-4338-b2a5-f730c732c1a4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.285846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb6a73-18f4-43b1-bb6f-528b6b59a50e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.294279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.295301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.295799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.296288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.297228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:12.297831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.299637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.314273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 435/850] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:12.319187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.328752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.329408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.329749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.330026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.331048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:12.331160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.332599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.352314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 416/851] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:12.363087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.416644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.436223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c40e8-8f42-49c9-9554-1c7ceda93eb8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.464071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.472850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.473450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.478081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.478081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.478561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9cdc-f051-4934-b8d0-49aef16a6eea None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.481472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.482921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:12.494895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:12.496076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:12.497399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 436/852] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:12.501528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.519072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.519983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.520444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.520909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.615214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.616798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.620965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a845b-c8ac-47e8-86df-d4b7fd84d44d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.657499 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.658026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.658475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.658750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.659532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:12.659833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.661343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.677904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.686389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 437/853] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:12.687729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.712035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.730658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 417/854] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:12.733527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.796008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e5ed-b66a-4bd8-81dc-7d9db54582da None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.832838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.833690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.834164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.834693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.844454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26a248-de4c-4039-9b60-6031a20ab6da None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:12.880945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:12.881496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:12.881777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:12.882064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:12.892758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.894415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.905808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:12.908177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:12.909762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:12.911141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:12.912256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:12.913435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:12.913435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:12.913435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:12.917037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 418/855] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:12.917928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:12.962625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.023416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 438/856] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:13.024656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.026471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ab46-0dfa-444c-9533-daa23c03f98c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.064190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.064801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.065079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.065351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.065997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.066333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.067849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.085614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 419/857] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:13.086730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.129832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.133455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb714fd-4fcc-451d-a427-c8d84e6826e4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.170608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.171587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.172013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.172436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.183977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.185714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.194907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.197940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac4d0e-1a7e-4b5b-ab5e-a1921edb83a2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:13.205209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:13.209954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:13.211126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:13.211126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:13.211126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:13.211868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 439/858] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:13.212735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.242635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.243270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.243545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.243812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.251753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.253247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:13.265410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:13.266655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:13.267568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:13.271281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 420/859] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:13.272144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.334501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcbd41-22d2-4ac9-96a6-ce9e322dc3c9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.376798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9e41c-c316-44b4-a7a3-4801ea9d5b53 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.387786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.388337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.388616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.388891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.389980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.390498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.392226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.410090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 440/860] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:13.412030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.423526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.424119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.424398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.424668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.425358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.425742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.427234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.430960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.447303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 421/861] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:13.449545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.471088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.535771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456156-b595-49ae-8f94-7e5bad69ded4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.555567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4a5a-6393-49b1-8e33-1a1f4d8e583f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.584084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.585141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.585425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.585696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.595227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.596868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.599113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.600514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.601243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.601970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.603867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=295dd5a7b7774239bf95df04555611f1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.604701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.607211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70c5dd-1534-486c-8587-a856d166776f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:13.608757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:13.609660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:13.610590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:13.614710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 441/862] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:13.615773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.618044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 422/863] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/role_assignments?user.id=295dd5a7b7774239bf95df04555611f1&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:13.660805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538300f-4414-4cb4-80a7-641cbfe7a38a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.661143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538300f-4414-4cb4-80a7-641cbfe7a38a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.661363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538300f-4414-4cb4-80a7-641cbfe7a38a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.663087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 423/864] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:13.670330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.670662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.670881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.696884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:13.698080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:13.706878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.741801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e342f2-a366-4396-b7ad-538dfa003efa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.784647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.785655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.786256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.786741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.787785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.788292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.790143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0227cac-50a2-48be-a08b-06ddceaafb01 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.804605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 424/865] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:13.806909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 442/866] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:13.813416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:13.832422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:13.923802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4d308-773d-493b-a307-b02b1e12d379 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:13.964174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:13.964814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:13.965040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:13.965270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:13.965994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=da9b976e48d3468b82d9168efced0b68) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:13.966330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:13.967828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f6237-86cb-46bb-a324-6beb0cb84f4e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:13.973968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 425/867] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:13 2022] GET /identity/v3/role_assignments?user.id=da9b976e48d3468b82d9168efced0b68&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:14.044463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c657eb-7551-431e-bb91-5fc95c2a56a7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.044744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c657eb-7551-431e-bb91-5fc95c2a56a7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.045016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c657eb-7551-431e-bb91-5fc95c2a56a7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.046627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 443/868] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:14.055936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.056382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.056662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.080525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:14.081806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c657eb-7551-431e-bb91-5fc95c2a56a7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:14.082406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:14.164906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 426/869] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:14.193374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c657eb-7551-431e-bb91-5fc95c2a56a7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:14.224484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:14.229475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:14.230136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.230574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.231028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.232172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:14.234022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:14.323685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 444/870] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:14 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:14.436536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:14.448024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:14.551519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d675bc0-5156-471d-a214-7af06a24af55 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:14.594812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:14.595965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.595965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.595965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.596545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:14.597904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:14.607605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:14.608623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:14.609554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 427/871] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Sep 9 11:26:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:14.614532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:14.720855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93302a8-f498-4a6d-88e0-00d11700a7a4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:14.763456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:14.764444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.764933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.765398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.766279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:14.766820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:14.768495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05060e-c1e3-45c1-aebb-dea59638ec36 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:14.773962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 445/872] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:14.780030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:14.850998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:14.887074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb10-7978-43d6-aa40-6b3391a7b8d6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:14.921368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:14.921979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:14.922289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:14.922571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:14.923199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:14.923502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:14.924875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944469f6-2f02-4720-bab3-eba07788ff4e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:14.931212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 428/873] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:16.039137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59801621-b9a7-4f0b-80e3-e27e936d7cba None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.039705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59801621-b9a7-4f0b-80e3-e27e936d7cba None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.039705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59801621-b9a7-4f0b-80e3-e27e936d7cba None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.041345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 446/874] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:16.047149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2718a4-87e5-4683-bff9-5639462a950b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.047562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2718a4-87e5-4683-bff9-5639462a950b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.047745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2718a4-87e5-4683-bff9-5639462a950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.066366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1e2718a4-87e5-4683-bff9-5639462a950b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:16.067542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1e2718a4-87e5-4683-bff9-5639462a950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:16.142430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 429/875] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:16.280628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654d8d-aae5-4f2b-b8cd-9bed4f1a57e5 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.280973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654d8d-aae5-4f2b-b8cd-9bed4f1a57e5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.281187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70654d8d-aae5-4f2b-b8cd-9bed4f1a57e5 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.282815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 447/876] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:16.288137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.288425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.288692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.307335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:16.309040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:16.371601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:16.377680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 430/877] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:16.447578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70654d8d-aae5-4f2b-b8cd-9bed4f1a57e5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:16.509797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:16.559969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70654d8d-aae5-4f2b-b8cd-9bed4f1a57e5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:16.594966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:16.595590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.595863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.596128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.606804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8326134e-fa2a-4ec6-9757-6b43de4cedc3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:16.612985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:16.613931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:16.614854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:16.618191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 448/878] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:16.620452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:16.626909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cd93a-efc4-4400-8924-e042e0d4f387 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.627193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cd93a-efc4-4400-8924-e042e0d4f387 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.627465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cd93a-efc4-4400-8924-e042e0d4f387 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.629054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 449/879] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:16.634179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.634464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.634741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.654931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:16.656356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:16.727729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 450/880] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:16.729387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:16.737686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b632a0-a428-4f30-b849-09a11d56f427 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:16.780197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:16.780934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.781288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.781656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.782440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:16.782812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:16.784266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:16.792968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 431/881] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:16.797235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:16.798257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:16.849779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c09d2-c502-46ef-9e18-aaa190efc667 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:16.873852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:16.888861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:16.889503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.889783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.890085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:16.890704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:16.892058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:16.908680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:16.909719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:16.910602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:16.914198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 451/882] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:16.916301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:16.936861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995e040-1a5d-4b33-940f-83a6429f03f7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:16.984323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:16.985514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:16.985988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:16.986477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.002250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.004248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.023808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac14fc-ce72-4ebb-87d2-682e944cc3d3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:17.024014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:17.024889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:17.026504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:17.026504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:17.026504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:17.026504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 432/883] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:17.026504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.039130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.065920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.066610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.066878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.067121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.067771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.068521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.069494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.075729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 452/884] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.081585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.133438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c19e-5ef7-4081-bc0c-6c285ae1f1d9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.141300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.168574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.169729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.170428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.171382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.172180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.173961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:17.183647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:17.184920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:17.186498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 433/885] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:17.187321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.195555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4d6c-d3e3-41e6-9c6b-cfb93eacef9c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.230124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.230772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.231052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.231322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.231957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.232264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.233643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.245012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 453/886] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.245803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.279177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.290647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38ebfb-e7e4-44c0-8aeb-9fc0c43cce72 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.323320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.323919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.324719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.324990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.325615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.325915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.327662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eeb43-4566-4e32-877b-4137131ffb4a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.331909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 434/887] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.333041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636849-93c0-4484-86a5-3fddc5f5e3bb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.333320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636849-93c0-4484-86a5-3fddc5f5e3bb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.333599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636849-93c0-4484-86a5-3fddc5f5e3bb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.335066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 435/888] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:17.335636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636849-93c0-4484-86a5-3fddc5f5e3bb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.352237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfad9b-718f-4c89-8bf9-21a9a057cd31 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.377548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.403008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.403008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.403008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.403008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.403587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.403587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.403587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5492bed-ad7f-40aa-803d-987409dfd28b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.413016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 454/889] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.413016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.413016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.413016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.430181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:17.431411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:17.466009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636849-93c0-4484-86a5-3fddc5f5e3bb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.503957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.504699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.505687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.505964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.510201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 455/890] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:17.511847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.516738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.518427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:17.532730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:17.533696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:17.534759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:17.536570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 436/891] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:17.543357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.549589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.635496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b098046-79a7-44f5-8255-b4c6e7a1539a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.662318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.669340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e76eca-d3f8-4c74-bc70-2e25557163e5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.681682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.684330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.684330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.684720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.685811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.686330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.688432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.700332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 456/892] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.702103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.712470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.713929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.714439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.715494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.716877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.717368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.719747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.746518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 437/893] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.747907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.812165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.821403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d01a98-4542-4993-8432-eceb44c676fa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.857819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.858906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.859310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.859587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.860407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] RBAC: Adding query filter params (interface=public, service_id=7fa5d89f22c3413eb9c87269c53328f2, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:17.860716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.862204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.863009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd369fa-5f68-4103-8726-16121b7db970 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:17.868663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 457/894] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/endpoints?service_id=7fa5d89f22c3413eb9c87269c53328f2&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:17.870907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:17.899885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:17.900494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:17.900767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:17.901038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:17.924386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:17.972344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:17.974280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:17.976187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1268e3-e4c3-41b9-adc7-030fade45638 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.011305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.012465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.012744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.013014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.013670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.015057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.076326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 438/895] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:17 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:18.077177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.113108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 458/896] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:17 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:18.113807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.140331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:18.144819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:18.196909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873460f-7292-4ade-aede-054689e2fdf9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.244839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.245509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.245788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.246688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.247339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.249384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.254069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a01cf4-e6f0-4bed-a03d-8d1de1754d42 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.256353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 439/897] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:26:18.297149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.298252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.298686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.299107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.310994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.312721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:18.329900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:18.331145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:18.332034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:18.332034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:18.332034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:18.333653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 459/898] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:18.338322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.466485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071be312-69e5-46b6-a1fb-57884f5829c5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.481345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:18.502652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.503326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.503601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.503874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.504516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:18.504883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.506217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.522423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 440/899] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:18.524159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.593930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:18.628299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f36f5-69f0-4ff9-840e-49a1065b6591 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.664055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.664697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.664980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.665256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.672355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.673868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:18.685485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:18.686406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:18.687350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 460/900] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:18.692611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.802866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf05a11-274e-4847-8063-f61c2970a4f9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:18.839166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:18.840342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:18.840628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:18.840898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:18.841591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:18.841894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:18.843358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dec0b2-9ae9-49eb-a015-26be9cf0a02a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:18.860932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 441/901] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:18.871812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:18.975144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ead194-d334-49a3-92b7-31da75e84b0f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:19.006412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:19.010497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:19.011410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:19.011735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:19.012010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:19.012747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] RBAC: Adding query filter params (role.id=c19cc115f7ef4256b96dfaf6512b4dff, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=295dd5a7b7774239bf95df04555611f1) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:19.013049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:19.014496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddc496-ca1e-411c-a9ee-412921a1e34d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:19.020424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 461/902] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:18 2022] GET /identity/v3/role_assignments?user.id=295dd5a7b7774239bf95df04555611f1&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=c19cc115f7ef4256b96dfaf6512b4dff => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:19.561627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229adb8-e15d-489d-8025-625f42daf496 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:19.562202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229adb8-e15d-489d-8025-625f42daf496 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:19.562202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229adb8-e15d-489d-8025-625f42daf496 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:19.563754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 442/903] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:19.569152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e5e45-77c4-4a0f-a702-541255194bd2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:19.569546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e5e45-77c4-4a0f-a702-541255194bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:19.569730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e5e45-77c4-4a0f-a702-541255194bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:19.588177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5d5e5e45-77c4-4a0f-a702-541255194bd2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:19.589380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5d5e5e45-77c4-4a0f-a702-541255194bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:19.656721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 462/904] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:19.942810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229adb8-e15d-489d-8025-625f42daf496 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:19.975761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:20.048499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229adb8-e15d-489d-8025-625f42daf496 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:20.088101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:20.088632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.088911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.089184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.089872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:20.091302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec7858-dae3-40db-979d-7ad64efd04fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:20.159855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 443/905] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:19 2022] POST /identity/v3/services => generated 241 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:20.417495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a091405-3e00-4a85-ad7e-689e813e31f1 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.417843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a091405-3e00-4a85-ad7e-689e813e31f1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.418125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a091405-3e00-4a85-ad7e-689e813e31f1 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.421634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 463/906] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:20.427114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d00f-3cbd-4f84-af05-575887e0a462 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.427402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d00f-3cbd-4f84-af05-575887e0a462 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.427672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370d00f-3cbd-4f84-af05-575887e0a462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.446414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e370d00f-3cbd-4f84-af05-575887e0a462 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:20.447598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e370d00f-3cbd-4f84-af05-575887e0a462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:20.530217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 444/907] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:20.602195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38930cc3-d0f0-41ea-8154-878c9cf3f253 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.602860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38930cc3-d0f0-41ea-8154-878c9cf3f253 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.602860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38930cc3-d0f0-41ea-8154-878c9cf3f253 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.604312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 464/908] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:20.609721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.610235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.610392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.629089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:20.630244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:20.644072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:20.704366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 445/909] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:20.808197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38930cc3-d0f0-41ea-8154-878c9cf3f253 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:20.909636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38930cc3-d0f0-41ea-8154-878c9cf3f253 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:20.937907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:20.950644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:20.951274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:20.951546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:20.951809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:20.952394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:20.954185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:20.963080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:20.964016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:20.964900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:20.969702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 465/910] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Sep 9 11:26:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:20.971063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:20.979091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.072210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84eb66-41a7-428b-9c7d-206680f7433a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.092807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b1d2c-9617-41f6-ab82-bbda1fa1d1cc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.113736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.114508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.114788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.115065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.115801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:21.116109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.117639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.124005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 446/911] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:21.130812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.137672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.138468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.138846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.139217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.140049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.141558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.146694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:21.151816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:21.153415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:26:21.154293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:21.162241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 466/912] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:21.162241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.221584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.251826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49157c88-97cb-45c5-a3b9-0a80e051642c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.285163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d078846-df3c-4200-9156-f5a3cfb6c62c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.287036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.287647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.287926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.288203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.288836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:21.289139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.290612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.300585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 447/913] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:21.301607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.323456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.324060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.324329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.324684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.325309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:21.325604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.326979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156657c-ac8e-4430-b46b-b60b81f928f1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.331865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 467/914] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:21.331865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9f8e-f69e-40fc-b4df-91f79bd8cda0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.331865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9f8e-f69e-40fc-b4df-91f79bd8cda0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.331865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9f8e-f69e-40fc-b4df-91f79bd8cda0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.338705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 468/915] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:21.338705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9f8e-f69e-40fc-b4df-91f79bd8cda0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.423701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0bae4-101b-4bb3-92de-af4f53c213eb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.443199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9f8e-f69e-40fc-b4df-91f79bd8cda0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.459090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.459288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.460052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.460347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.460623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.461329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] RBAC: Adding query filter params (interface=public, service_id=d020838c69024a68bc87097d01eed6f3, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:21.461639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.463095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea1446-5f90-46fe-9cbc-b849492028e9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.470604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 448/916] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/endpoints?service_id=d020838c69024a68bc87097d01eed6f3&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:21.471810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.472097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.472426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.486461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.487347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.487810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.488274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.489080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:21.489556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.490688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:21.491431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.491892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:21.497265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 469/917] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:21.498295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.574095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 449/918] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:21.574095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.619443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c74d31-79a2-434c-9abe-4008cf6aa383 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.655488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.660294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.660915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.661187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.661455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.662076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.663570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.670682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 470/919] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:26:21.688152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a2988-5d08-47b7-9976-30bac47f3636 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:21.733246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:21.733878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:21.734185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:21.734523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:21.735216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:21.736639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:21.826319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 450/920] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:21.854796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:21.907357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:21.974823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0befd6-768a-4085-a933-7d4ce74580bc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.020829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.021470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.021749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.022020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.033648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:22.038549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:22.040649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:22.042642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 471/921] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:22.048311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.160300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7e1d0-1f68-4ca3-9df9-b4501288ac23 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.188946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:22.198195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.199092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.199690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.200162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.201029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:22.201527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:22.203397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65548571-1f95-4756-aaaf-4124c797f969 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:22.212539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 451/922] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:22.218635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.337522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:22.341914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a85aad-29a4-431b-84a7-f076f21b5995 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.378715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.379726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.379726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.379873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.391637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:22.393080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:22.408160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:22.409138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:22.410035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:22.410035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:22.410035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:22.410035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 472/923] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:22.412479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89004d-d477-4527-af67-1657ccf3a77c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.412871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89004d-d477-4527-af67-1657ccf3a77c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.413245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89004d-d477-4527-af67-1657ccf3a77c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.414920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 452/924] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:22.419755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.420336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.420615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.420877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.438653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:22.440267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:22.530140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 453/925] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:22.541008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d1c1-224b-494b-b090-c84007dea827 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.581475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.582514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.582995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.583531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.584381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:22.584883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:22.586507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:22.599980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 473/926] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:22.606083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.720559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093c40-f2d1-431a-9d39-eefde1c0c962 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.738900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:22.755611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.756314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.756592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.756861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.764046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:22.765480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:22.777261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:22.778772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:22.779739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 454/927] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:22.785032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.798914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:22.831474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:22.903517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e6359-4027-4d37-9e0d-c396e771a368 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.944618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:22.946202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:22.946797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:22.947349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:22.948366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:22.948886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:22.951092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:22.952486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ad5-2d72-4e8a-8413-8be4600bad75 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:22.974316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 474/928] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:22.979746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.001209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.002783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.003530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.004019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.005080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.007382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:23.019920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:23.020916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:23.021766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:23.026095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 455/929] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:23.029777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.061810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.084434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.100340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720fdf-8d5c-4699-a174-b6b40a3ff3e2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.138623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.139529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.139956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.140373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.151558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.153306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.154914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcd59-3f5f-4ebf-890c-efd1a2ac5723 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:23.169666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:23.170706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:23.171609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:23.171609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:23.171609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:23.178281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 475/930] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:23.179899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.193138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.194048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.194353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.194621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.195271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:23.195806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.197180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.206142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 456/931] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:23.210912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.311215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe066f8-0c66-44ad-aa71-ee67b29424b3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.315132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a28-d37f-4699-9900-b06e8f21ec6e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.321661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.338530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.350544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.351119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.351398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.351674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.352398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:23.352595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.353997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.358319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.359243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.359688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.360118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.360946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:23.361420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.362169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 457/932] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:23.367614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.367776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.392921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 476/933] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:23.394796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.495172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6833c-432e-46fe-91f0-290114bb5d98 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.505895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5a66-faba-4291-a887-40d395576900 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.539006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.540451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.541190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.541674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.542780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] RBAC: Adding query filter params (interface=public, service_id=59ba137489c04f348b6ad1e2c603d6fa, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:23.543731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.545534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.548788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.549625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.550125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.550463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.552219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 458/934] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/endpoints?service_id=59ba137489c04f348b6ad1e2c603d6fa&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:23.557661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.560583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.562293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:23.574267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:23.575275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:23.576179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:23.583444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 477/935] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:23.585114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.596183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.631002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.668252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737b470-4647-4f0f-be3d-5441ae59d739 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.692242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ad687-028a-4856-be25-b4da40d6a033 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.705615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.706320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.706594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.706863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.707567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.709114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.722484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 459/936] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:26:23.732705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.733713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.734228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.735365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.735918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:23.736481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.738203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592395c-8cf8-4bd8-8f6d-5bf8571ab9ef None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.758262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 478/937] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:23.759994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:23.879938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:23.884223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03ce62-c9b9-42ff-8682-32522954940f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:23.920446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:23.921109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.921384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.921658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/295dd5a7b7774239bf95df04555611f1/roles/c19cc115f7ef4256b96dfaf6512b4dff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.923815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=295dd5a7b7774239bf95df04555611f1, role_id=c19cc115f7ef4256b96dfaf6512b4dff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:23.925269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88cbed-f317-4ad3-8017-5942b273b8d1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:23.960119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ebfe-b932-4e36-994b-1b0892660331 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.960650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ebfe-b932-4e36-994b-1b0892660331 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.960650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ebfe-b932-4e36-994b-1b0892660331 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.963134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 479/938] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:23.968267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47291-857f-4b07-a830-b081e632a0e7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:23.968557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47291-857f-4b07-a830-b081e632a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:23.968832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47291-857f-4b07-a830-b081e632a0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:23.987127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8ad47291-857f-4b07-a830-b081e632a0e7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:23.988320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8ad47291-857f-4b07-a830-b081e632a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:24.042944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 460/939] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:23 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/295dd5a7b7774239bf95df04555611f1/roles/c19cc115f7ef4256b96dfaf6512b4dff => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:24.071511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 480/940] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:24.167652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f5f0-b14d-411f-8e09-233cf1e049da None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.168007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f5f0-b14d-411f-8e09-233cf1e049da None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.168193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f5f0-b14d-411f-8e09-233cf1e049da None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.169863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 461/941] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:24.175629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.175957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.176177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.196433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:24.197645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:24.225262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:24.275116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 481/942] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:24.354675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f5f0-b14d-411f-8e09-233cf1e049da None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.474120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f5f0-b14d-411f-8e09-233cf1e049da None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:24.511647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:24.513179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.513682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.514203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.515012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:24.516589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:24.526216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:24.527238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:26:24.528292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.531119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 462/943] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Sep 9 11:26:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:24.539270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.547041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.573302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:24.661630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf06aa-0b25-4075-aa6e-1b650ef46fc7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:24.661840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e920-8718-4c61-b786-4fab055a0c16 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:24.708100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:24.708712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.708988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.709242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:24.709369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.709882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:24.709981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.710208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:24.710351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.710787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.711502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:24.711605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:24.713778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:24.716869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING keystone.server.flask.common [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 11:26:24.717008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 482/944] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:24.722604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.751816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:24.764762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.sql.core [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:26:24.764762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 11:26:24.764762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [parameters: {'id': '64b972a00b0a4c4fbd734dc47f8c4e29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 11:26:24.764762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81665) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64b972a00b0a4c4fbd734dc47f8c4e29.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64b972a00b0a4c4fbd734dc47f8c4e29. Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:26:24.773823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 11:26:24.776763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application yield resource Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise self.value Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application session.commit() Sep 09 11:26:24.777782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:26:24.779412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:26:24.780420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:26:24.782227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [parameters: {'id': '64b972a00b0a4c4fbd734dc47f8c4e29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.784356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:24.786868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64b972a00b0a4c4fbd734dc47f8c4e29. Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 463/945] 10.222.0.60 () {60 vars in 1140 bytes} [Fri Sep 9 11:26:24 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:24.790974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.847615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbade-5fa1-46c9-b042-a105605fbfd5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:24.891768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:24.893101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.893532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.894000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.894844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:24.895624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:24.896944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:24.897324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:24.909786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 483/946] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:24.912449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:24.933508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8014-aa6e-4766-8b3a-964e2761579d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:24.977006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:24.977919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:24.978378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:24.978796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:24.986077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:24.987900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:25.000419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:25.001334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:25.002477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:25.004522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 464/947] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:25.013063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:25.022338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:25.039574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6249aa4-7ca4-4835-870b-cafcdde13903 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:25.083223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:25.084218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:25.084640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:25.085053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:25.085862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:25.088306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcd679-7292-4d55-beba-b23257408565 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:25.124451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f495-e558-4706-ad0b-ca87b8d9180c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:25.167189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:25.167838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:25.168164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:25.168436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:25.169844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:25.170117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:25.171656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af9089-4ea6-4f49-94c8-cc97f265cccc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:25.176357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 484/948] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:25.187353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:25.201360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 465/949] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:26.090269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f35c-5cb6-4452-a887-0969c349b9a5 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.090849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f35c-5cb6-4452-a887-0969c349b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.090849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f35c-5cb6-4452-a887-0969c349b9a5 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.092551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 485/950] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:26.098886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656b617-a86a-47ac-ae1b-397027039838 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.099402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656b617-a86a-47ac-ae1b-397027039838 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.099402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656b617-a86a-47ac-ae1b-397027039838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.118448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e656b617-a86a-47ac-ae1b-397027039838 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:26.119954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e656b617-a86a-47ac-ae1b-397027039838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:26.196706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 466/951] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:26.372694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0afdb-8e08-4e04-9242-15d00cae4d0e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.373171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0afdb-8e08-4e04-9242-15d00cae4d0e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.373635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0afdb-8e08-4e04-9242-15d00cae4d0e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.377717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 486/952] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:26.383245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.383245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.383527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.402321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:26.403573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:26.469403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0afdb-8e08-4e04-9242-15d00cae4d0e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:26.474028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 467/953] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:26.575591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0afdb-8e08-4e04-9242-15d00cae4d0e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:26.610151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:26.610778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.611058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.611333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.611925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:26.613279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:26.622676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:26.623844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:26.625537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 487/954] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:26.635147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:26.694651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:26.739497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86def409-65f2-4fb6-8efd-bc9af4479cce None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:26.766421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:26.778608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:26.779277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.780244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:26.780649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.780941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.781660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:26.782287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:26.784051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:26.790973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 468/955] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:26.797621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:26.893135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16012e-f1a4-437b-b7af-8fa63788b28b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:26.920539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d399a-f37e-4137-9e99-b1d41aefbab8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:26.935318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:26.935949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.936221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.936486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.938446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:26.948709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:26.954639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:26.955718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:26.956831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:26.957667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:26.957667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:26.957667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:26.957667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:26.957995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:26.958085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:26.959446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:26.959771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 488/956] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:26.965643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:26.968894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 469/957] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:26.974574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.058471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:27.088202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23fad8-aeff-49f9-8a2a-f25ab6f1fe40 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.107762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b689e7e-eae7-409c-bf23-c6c5a00b8f18 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.124799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.125498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.125857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.126277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.127179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.128959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.155654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.155654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.155654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.155654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.156217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:27.156217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.156217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.164411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 489/958] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:27.170244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.222851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 470/959] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:26:27.291582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b19618-9924-49d2-8f5d-93a41e69743e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.327409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:27.340501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.341203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.341476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.341738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.353418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.355027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-6b0fce35-e87d-4bdf-a3f6-2ab0a78d8a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:27.371446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:27.373021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:27.373956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:27.373956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:27.373956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:27.373956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 490/960] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:27.379465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.425125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:27.426688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdbd7-9115-42ce-aa68-ea46b98550cf None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.426971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdbd7-9115-42ce-aa68-ea46b98550cf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.427245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdbd7-9115-42ce-aa68-ea46b98550cf None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.428591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 491/961] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:27.435685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3718e-f669-4061-af02-6f4ad230a8f7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.442234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3718e-f669-4061-af02-6f4ad230a8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.442234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3718e-f669-4061-af02-6f4ad230a8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.458121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-04d3718e-f669-4061-af02-6f4ad230a8f7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:27.459371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-04d3718e-f669-4061-af02-6f4ad230a8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:27.505425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d38e6-86d5-4341-8886-161d4295e5a8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.539449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 492/962] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:27.553921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.554570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.554851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.555127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.555775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:27.556079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.557457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.563739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b0f3-e26c-4ed2-bde9-7881bf1bdf76 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.564024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b0f3-e26c-4ed2-bde9-7881bf1bdf76 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.564298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b0f3-e26c-4ed2-bde9-7881bf1bdf76 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.565609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 493/963] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:27.570590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 471/964] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:27.570849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811956-8523-4056-9835-2f3638153b4a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.571155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811956-8523-4056-9835-2f3638153b4a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.571426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811956-8523-4056-9835-2f3638153b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.579294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.595001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-38811956-8523-4056-9835-2f3638153b4a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:27.596466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38811956-8523-4056-9835-2f3638153b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:27.673985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 494/965] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:27.690394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:27.694828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ac55d-26aa-4bf6-99c6-6a44858da8f2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.730505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.731139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.732012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.732283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.739660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.741193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:27.753825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:27.754821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:27.755786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 472/966] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:27.761071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38811956-8523-4056-9835-2f3638153b4a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.803620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:27.835941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.869572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38811956-8523-4056-9835-2f3638153b4a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.910252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.911016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.911297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.911564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.912347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:27.912657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.914226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:27.933058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 495/967] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:27.939441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:27.957355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573f79-7470-47b8-a9f5-8c124cb39257 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:27.993854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:27.994583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:27.994855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:27.995121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] PATH_INFO: `/v3/services/compute` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:27.995694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:27.997457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:28.005907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:28.007017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.007882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 473/968] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:28.008712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.035474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.046203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.052384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd7b7-e878-48a7-87f2-5842bba46977 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.088238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.088845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.089195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.089489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.122648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eca3f-2471-4747-8235-5e300fc8e8df None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.160419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.160926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.161311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.161391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.161625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.161846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.162270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.163686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.203475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.sql.core [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:26:28.203475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 11:26:28.203475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [parameters: {'user_id': '5d4f64de52314158bab7fbcb8309e47a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 11:26:28.203475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81665) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:26:28.210277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:26:28.211150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application yield resource Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise self.value Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application session.commit() Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:26:28.212172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:26:28.213082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:26:28.213991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:26:28.214932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [parameters: {'user_id': '5d4f64de52314158bab7fbcb8309e47a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:28.215855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 11:26:28.218749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 474/969] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Sep 9 11:26:28 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:28.220864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.269050 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.280699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 496/970] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:28.280699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.306752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.321835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6186a-c57a-49ae-99f9-a9b7904ab9da None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.359867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.361379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.361379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.361379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.362245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:28.362716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.364287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.371807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 475/971] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:28.372616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.399566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480e3d-d723-47c6-abb4-e9b0a02afc45 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.436555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.437180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.437458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.437729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.444697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.446173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:28.457794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:28.458903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 497/972] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:28.459863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.494217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d1b32-2217-4d2d-8b55-442bcc304680 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.523538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.531291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.539277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.539277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.539277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.543337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.544902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.553201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:28.560990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:28.565122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:28.566153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:28.566153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:28.566153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:28.566153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 476/973] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:28.567649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.571855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924703d7-1a86-428e-8c2d-7dc4523b3b39 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.606629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.607244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.607524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.607800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.608439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:28.608740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.610123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.626085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 498/974] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:28.626085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.688443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac25b5-d598-49a1-bb24-3c685008a74e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.728387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.729163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.729534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.729958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.730836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:28.731353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.733891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.769588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65b296-03d2-4252-8dda-4899e6c82211 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.775054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 477/975] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:28.794546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.815843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.816577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.816925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.817262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:28.817987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:28.818377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:28.819912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:28.834009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 499/976] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:28.839587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:28.911257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:28.961938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038ba34-12fb-4de5-92f7-07e901a6d17f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:28.996609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:28.997307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:28.997578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:28.997843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:29.004966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:29.006443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:29.018377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:29.019319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:29.020687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 478/977] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:29.025697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:29.131958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c56-9d46-4ca9-a31a-76f40b006fbb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:29.167980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:29.168624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:29.168837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:29.169129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:29.169809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:29.170136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:29.171538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42459277-685a-412f-8eab-3f3aaf2326de None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:29.183544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 500/978] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:29.189396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:29.289269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b04957-55ed-4821-8e52-0c07784b8a69 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:29.311521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:29.330136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:29.333693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:29.333693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:29.333693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:29.333693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] RBAC: Adding query filter params (role.id=c19cc115f7ef4256b96dfaf6512b4dff, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=295dd5a7b7774239bf95df04555611f1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:29.333693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:29.334142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cb49a-6c1e-4ecd-a758-f570f13cfee9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:29.339940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 479/979] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:29 2022] GET /identity/v3/role_assignments?user.id=295dd5a7b7774239bf95df04555611f1&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=c19cc115f7ef4256b96dfaf6512b4dff => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:29.563109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8edaf-d5a8-4ced-9a2f-02b8fc134bc5 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:29.563314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8edaf-d5a8-4ced-9a2f-02b8fc134bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:29.563574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8edaf-d5a8-4ced-9a2f-02b8fc134bc5 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:29.564961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 501/980] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:29.571257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:29.571695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:29.572092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:29.592208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:29.593914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:29.664453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 480/981] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:29.930459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8edaf-d5a8-4ced-9a2f-02b8fc134bc5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:29.974732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:30.034112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8edaf-d5a8-4ced-9a2f-02b8fc134bc5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:30.067975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:30.068722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.069064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.069428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.070101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:30.071552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:30.081302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:30.082550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:30.083636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:30.085053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 502/982] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:30.089986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:30.190471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9d6d5-6d76-437f-b1d1-ffdacb12bad4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:30.203366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:30.224317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:30.224924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.225198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.225463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.226103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:30.226407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:30.227774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:30.233292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 481/983] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:30.239070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:30.348570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4237d-d7c0-45ef-99c8-91213650af04 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:30.350910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:30.388274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:30.388880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.389152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.389417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.390040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:30.390366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:30.391700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:30.396686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 503/984] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:30.402504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:30.507647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82721c-3d35-4e85-a853-44110d2cefad None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:30.540912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:30.541509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.541782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.542046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.542769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] RBAC: Adding query filter params (interface=admin, service_id=59ba137489c04f348b6ad1e2c603d6fa, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:30.543072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:30.544983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9254ad-09fb-4062-a62d-7826d1ba18dc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:30.550009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 482/985] 10.222.0.60 () {56 vars in 1249 bytes} [Fri Sep 9 11:26:30 2022] GET /identity/v3/endpoints?service_id=59ba137489c04f348b6ad1e2c603d6fa&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:30.555616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:30.655183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d024a-14ad-46fb-8a09-d53f31dbc0ea None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:30.697062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:30.697675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.697952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.698251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.698855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:30.700221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1674bb1-15e2-4a32-96cd-0d9f899371a0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:30.714090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 504/986] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:30 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:26:30.877568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb112-00c3-4646-a94a-867d43215ae8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.877851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb112-00c3-4646-a94a-867d43215ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.878150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fb112-00c3-4646-a94a-867d43215ae8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.879515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 483/987] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:30.884666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619902-8e25-4e98-a179-52b95554f0f8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:30.884949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619902-8e25-4e98-a179-52b95554f0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:30.885219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23619902-8e25-4e98-a179-52b95554f0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:30.904128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-23619902-8e25-4e98-a179-52b95554f0f8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:30.905303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-23619902-8e25-4e98-a179-52b95554f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:30.932537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:30.974162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 505/988] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:31.245343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fb112-00c3-4646-a94a-867d43215ae8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:31.289423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:31.346914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fb112-00c3-4646-a94a-867d43215ae8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:31.389257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:31.389867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:31.390174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:31.390446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:31.391108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:31.392529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515ccb4-8d08-4478-9381-fab2bb95ff4c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:31.486716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 484/989] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:31 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:31.546154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1c02a-ce9c-4ba5-ac62-1b86c9863b8c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:31.546728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1c02a-ce9c-4ba5-ac62-1b86c9863b8c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:31.546728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1c02a-ce9c-4ba5-ac62-1b86c9863b8c None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:31.548146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 506/990] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:31.553872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:31.554214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:31.554466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:31.573391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:31.574651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:31.658086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 485/991] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:31.926668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1c02a-ce9c-4ba5-ac62-1b86c9863b8c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:32.031317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1c02a-ce9c-4ba5-ac62-1b86c9863b8c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:32.064363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:32.064986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:32.065266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:32.065537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:32.066230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:32.067513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:32.076716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:32.077696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:32.079103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 507/992] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Sep 9 11:26:31 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:32.083964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:32.111241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:32.182175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ab451-3f43-438b-82a8-2f0905cee45e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:32.217433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:32.218043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:32.218349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:32.218622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:32.219251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] RBAC: Adding query filter params (name=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:32.219553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:32.220925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aca8b9-a1b5-4f1f-892b-7082d11541bd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:32.230352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 486/993] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:32.232579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:32.261937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:32.332770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b5ab8-87a3-4e68-ae21-2506cf23c6d0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:32.370093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:32.370701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:32.370978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:32.371245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:32.371870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:32.372165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:32.374178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88826b-92ff-43cb-bca5-964d11a2c5e6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:32.381616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 508/994] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:32.895485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98c8-7efe-44f0-a21e-07673a886034 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:32.896036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98c8-7efe-44f0-a21e-07673a886034 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:32.896036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98c8-7efe-44f0-a21e-07673a886034 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:32.897496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 487/995] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:32.902644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:32.903067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:32.903213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:32.921980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:32.923201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:32.988638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 509/996] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:33.272496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d98c8-7efe-44f0-a21e-07673a886034 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:33.379311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:33.389400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d98c8-7efe-44f0-a21e-07673a886034 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:33.431581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:33.432217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.432492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.432756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.433527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:33.435791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:33.444784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:33.445818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:33.451754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 488/997] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Sep 9 11:26:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:33.452855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:33.479560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:33.571528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e499d5-e35c-48ea-8a83-8bef35c84423 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:33.610559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:33.611194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.611474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.611745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.612431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:33.612733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:33.614142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deccba-e68a-4012-94fe-9ceee4aaa71a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:33.619674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 510/998] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:33.626271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:33.705873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51536310-c299-47b8-903d-899ef7b174f3 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.706176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51536310-c299-47b8-903d-899ef7b174f3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.706887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51536310-c299-47b8-903d-899ef7b174f3 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.707831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 511/999] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:33.712660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.712941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.713211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.729177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3376a1b-15c5-4443-a3a0-c98d1f5db7a0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:33.735921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:33.735921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:33.765919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:33.766557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.766834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.767108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.767735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:33.768037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:33.769446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:33.774442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 489/1000] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Sep 9 11:26:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:33.779873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:33.800038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:33.801700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 512/1001] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:33.879391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4ff0c-7d78-4fee-bb02-8de25f3ca176 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:33.912284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:33.912923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:33.913150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:33.913420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:33.914102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:33.915467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:33.988016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 490/1002] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:26:34.071365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.087520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:34.186204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42267a-e80a-4db8-9ebf-4f29ea181c94 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:34.226489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:34.227139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.227413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.227675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.228248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:34.229583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:34.238226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:34.239236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:34.240767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 513/1003] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:34.245356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.322867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:34.346537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2931a4d-55af-4c5e-97ff-52b73f39fead None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:34.389678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:34.390318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.390601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.390874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.391501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:34.391805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:34.393166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f6a5b-e255-4c49-b869-3893ab2f2e92 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:34.402082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 491/1004] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:34.403944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.488701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:34.512358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fad659-d253-4976-b71a-880f29712f03 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:34.548776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:34.549397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.549671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.549941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.550608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:34.550906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:34.552284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:34.557310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 514/1005] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:34.560368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93fbea-1239-4cd0-aab8-a3d69a36444b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.560653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93fbea-1239-4cd0-aab8-a3d69a36444b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.560928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93fbea-1239-4cd0-aab8-a3d69a36444b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.562429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 492/1006] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:34.562676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.567048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6851e-5665-499e-a114-d092b8066659 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.567330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6851e-5665-499e-a114-d092b8066659 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.567606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6851e-5665-499e-a114-d092b8066659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.586755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-80a6851e-5665-499e-a114-d092b8066659 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:34.587937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-80a6851e-5665-499e-a114-d092b8066659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:34.657037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 493/1007] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:34.670298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ea7ac-f203-4662-93a9-0c6cecb04b69 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:34.704924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:34.705638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] RBAC: Adding query filter params (interface=public, service_id=ec778cbd5ffe4dceaf189b72d1a0b556, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:34.744327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 515/1008] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:34 2022] GET /identity/v3/endpoints?service_id=ec778cbd5ffe4dceaf189b72d1a0b556&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:34.744843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6851e-5665-499e-a114-d092b8066659 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.793266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:34.822213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6851e-5665-499e-a114-d092b8066659 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:34.857950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:34.858571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:34.858848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:34.859114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:34.859713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:34.861138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4f856-b3ba-488b-8a2e-27b66861042c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:34.874335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 494/1009] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:34 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:26:34.928817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:34.931982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:35.037071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120d082-ff22-4ba3-a906-25f983da40be None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:35.072246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:35.073607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:35.074135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:35.074622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:35.075470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:35.077583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b9e5-9487-45fd-b965-218a6ada8638 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:35.149756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 516/1010] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:34 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:36.169014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb9-6f5a-40b4-b0de-b88a6e79c433 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:36.169610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb9-6f5a-40b4-b0de-b88a6e79c433 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:36.169718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb9-6f5a-40b4-b0de-b88a6e79c433 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:36.171196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 495/1011] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:36.176241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:36.176617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:36.176796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:36.195669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:36.196852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:36.255120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:36.271216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 517/1012] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:36 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:36.538444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823deb9-6f5a-40b4-b0de-b88a6e79c433 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:36.641829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823deb9-6f5a-40b4-b0de-b88a6e79c433 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:36.676142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:36.676761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:36.677032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:36.677298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:36.678414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:36.679836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68da35-e7e1-4728-a487-b45f1fb9a8d8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:36.681556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 496/1013] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:36.687091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:36.785443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560aae-627b-4e56-abb9-7c021cf7c7d0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:36.819549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:36.820081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:36.820355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:36.820623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:36.821270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:36.822706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e1bf4-1bf0-4617-b4fa-83aa210eee03 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:36.934096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 518/1014] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:26:36 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:37.044793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd95081-6b7e-4c31-a321-4024c33c3561 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.045127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd95081-6b7e-4c31-a321-4024c33c3561 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.045345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd95081-6b7e-4c31-a321-4024c33c3561 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.046920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 497/1015] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:37.052478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab45bc4-78c7-4a2e-8da4-339eef775c3f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.052795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab45bc4-78c7-4a2e-8da4-339eef775c3f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.053041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab45bc4-78c7-4a2e-8da4-339eef775c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.072620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6ab45bc4-78c7-4a2e-8da4-339eef775c3f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:37.073759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6ab45bc4-78c7-4a2e-8da4-339eef775c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:37.112802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:37.147905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 519/1016] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:37 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:37.370296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716163-f921-455e-8d2b-d06584558789 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.370814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716163-f921-455e-8d2b-d06584558789 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.370898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716163-f921-455e-8d2b-d06584558789 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.372304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 498/1017] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:37.378178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.378602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.378774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.396635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:37.397812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:37.415556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716163-f921-455e-8d2b-d06584558789 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.420623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:37.468410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 520/1018] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:37 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:37.525480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716163-f921-455e-8d2b-d06584558789 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:37.559907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:37.560568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.560907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.561179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.561756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:37.563204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:37.572012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:37.573034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:26:37.573947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:37.579060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 499/1019] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:37.580240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.684906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:37.688333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2337-7a18-412b-a15a-be8ad023154f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:37.722251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:37.722860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.723138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.723412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.724041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:37.724341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:37.725754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:37.730823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 521/1020] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:37.736933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.745682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.804198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:37.837987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a386387-f2e3-4b20-881d-031255708d8e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:37.852481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eafeb1-d161-4d1f-8ceb-5aeef2cd78a2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:37.876738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:37.877348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.877627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.877903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.878576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:37.878882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:37.880247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:37.889834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 500/1021] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Sep 9 11:26:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:37.890857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.893157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:37.893830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:37.894150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:37.894368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:37.894962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:37.896320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:37.905915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:37.906954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:37.907828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 522/1022] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Sep 9 11:26:37 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:37.912792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:37.944473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:38.000893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8f438-913d-4e79-b2e6-8b649cdf8fc4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:38.023640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43231c2b-3240-42d4-9ae7-c61fe31ad9b7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:38.031038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:38.035918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:38.036523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:38.036877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:38.037162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:38.037726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:38.039128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:38.059992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:38.060658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:38.060934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:38.061201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:38.061832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:38.062156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:38.063500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:38.068252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 523/1023] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:38.073850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:38.163965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:38.172844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453cfee-fc73-4913-810d-179486e8f4c5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:38.186994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 501/1024] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:26:38.230699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:38.231232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:38.231510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:38.231780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:38.232918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] RBAC: Adding query filter params (type=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:38.233219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:38.234680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:38.245755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 524/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:38.247174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:38.354946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59517122-37b5-47ba-8490-b02c61aa0f04 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:38.397371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:38.397989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:38.398306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:38.398582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:38.399288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] RBAC: Adding query filter params (interface=public, service_id=3039866bdbf44dcf975310b41231b19e, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:38.399594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:38.400964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331acf1-a63c-471d-b712-42d38d0efade None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:38.406790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 502/1026] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:26:38 2022] GET /identity/v3/endpoints?service_id=3039866bdbf44dcf975310b41231b19e&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:38.412766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:38.521387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9f0f1-cb46-4f12-9e9a-04577132f051 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:38.555275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:38.555883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:38.556161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:38.556433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:38.557038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:38.558409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846c803-2cb6-44e9-b777-622685304de5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:38.573937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 525/1027] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:26:38 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:26:39.141568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c966e-a2d7-4eb7-bde9-30f12484d6ed None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:39.141921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c966e-a2d7-4eb7-bde9-30f12484d6ed None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:39.142139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c966e-a2d7-4eb7-bde9-30f12484d6ed None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:39.143629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 503/1028] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:39.148596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:39.148884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:39.149154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:39.167470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:39.168643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:39.179647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:39.241389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 526/1029] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:39.505328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c966e-a2d7-4eb7-bde9-30f12484d6ed None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:39.559997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:39.611701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c966e-a2d7-4eb7-bde9-30f12484d6ed None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:39.645545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:39.646195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:39.646486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:39.646727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:39.647818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:39.649295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8436-ea43-4e64-bb85-61da34668a8a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:39.651076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 504/1030] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:39.656648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:39.760752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb758-5d16-49d9-ab90-1e6954853881 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:39.794706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:39.795306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:39.795577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:39.795841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:39.796493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:39.797888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0b354-0755-490a-8462-395b7080289e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:39.874088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 527/1031] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:26:39 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:40.478722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe434a4-c573-4867-bb52-976e0ed04aca None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:40.479071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe434a4-c573-4867-bb52-976e0ed04aca None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:40.479276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe434a4-c573-4867-bb52-976e0ed04aca None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:40.480767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 505/1032] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:40.486398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ff73-c510-4aa4-8166-f056ad61ed0d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:40.486689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ff73-c510-4aa4-8166-f056ad61ed0d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:40.486961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ff73-c510-4aa4-8166-f056ad61ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:40.493162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:40.505612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e485ff73-c510-4aa4-8166-f056ad61ed0d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:40.506806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e485ff73-c510-4aa4-8166-f056ad61ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:40.579239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 528/1033] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:40.729707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876351-2e43-44d1-aaa4-960e2ce39066 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:40.730275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876351-2e43-44d1-aaa4-960e2ce39066 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:40.730275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df876351-2e43-44d1-aaa4-960e2ce39066 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:40.731545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 506/1034] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:40.737107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:40.737515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:40.743245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:40.756193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:40.757382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:40.828414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 529/1035] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:40.848240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df876351-2e43-44d1-aaa4-960e2ce39066 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:40.948290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df876351-2e43-44d1-aaa4-960e2ce39066 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:40.967782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:40.981554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:40.982129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:40.982939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:40.983205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:40.993354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:41.000674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:41.001907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:41.002992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:41.010441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 507/1036] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:41.011433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.071608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.094816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.111347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7624c8e-8127-4c4a-9c47-fc3309df0719 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.150873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.151602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.151884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.152159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.152825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:41.153127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.154790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.168444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 530/1037] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:41.169590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.203689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7196fb-346e-41bc-b164-40fb6a376a4c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.236982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.237528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.237801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.238082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] PATH_INFO: `/v3/services/image` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.238669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.240029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:41.248542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:41.249723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:41.250789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 508/1038] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:41.255348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.279046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094b751-c337-4035-b02a-58ded4c30705 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.303324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.307953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.313524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.314179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.314465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.314739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.325324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.327418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:41.342376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:41.343354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:41.344328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:41.344328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:41.344328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:41.344328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 531/1039] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:41.349295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.366466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3020-27d7-439d-aa81-cb403679c55a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.410518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.411121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.411683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.411791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.412388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] RBAC: Adding query filter params (name=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:41.412691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.414172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.419433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 509/1040] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:41.424890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.460587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87dc33-15ba-4010-8655-d4009c04e641 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.495318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.495852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.496128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.496397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.497037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:41.497346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.499243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.511160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 532/1041] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:41.516544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.530102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacd12c-0f23-4dfd-b5ce-07c6575ed835 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.539112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.554495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.568428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.569125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.569411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.569684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.570360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] RBAC: Adding query filter params (type=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:41.570675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.572020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.577805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 510/1042] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:41.584221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.632745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4459-af8e-422f-bb69-13edf586b974 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.667436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.668053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.668332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.668602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.676069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.677498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.684995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52a3e0-2040-4706-944b-4f6e81b6cdda None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:41.689716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:41.690902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:41.692280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:41.695823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 533/1043] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:41.696446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.722323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.722933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.723227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.723483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.724150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.725513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.798891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.804418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68f887-1f24-439a-896f-ebe558f33126 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:41.834088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 511/1044] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:26:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:26:41.847193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:41.847821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:41.848099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:41.848373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:41.849120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:41.849428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:41.850926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b6f0-63f0-4e79-b667-f0595234ce94 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:41.876622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 534/1045] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:41.878117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:41.909757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:41.991670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf417e2-4e2d-4ed0-936c-6ab4e72b77fe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.036132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.036741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.037045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.037315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.106543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea983f9-d720-4489-88c9-4d283cb5901c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.106842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea983f9-d720-4489-88c9-4d283cb5901c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.107018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea983f9-d720-4489-88c9-4d283cb5901c None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.108145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.110094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 535/1046] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:42.110251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.113453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.113730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.114000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.134966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:42.137163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:42.138633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.208599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 536/1047] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:42.221504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 512/1048] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:42.222658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.301300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.321072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c682b-6c17-4e6f-bbb0-3c7955fb9c00 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.357561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.358138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.358474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.358748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.379641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.381282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:42.398640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:42.399756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:42.400758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:42.400758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:42.400758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:42.402409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 537/1049] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:42.407131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.486107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.510941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49968b5-a158-40a8-9889-5076c863328d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.539511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.548509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.549186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.549471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.549744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.550421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:42.550727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.552096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.564281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 513/1050] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:42.569776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.592411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5a94-ad6d-440d-b461-4e87e44b14bb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.624175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.630255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.630935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.631215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.631489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.642656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:42.647507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:42.648486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:42.649353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:42.653676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 538/1051] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:42.654476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.676980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8364490-f1a0-4969-8482-7d5eb65660db None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.711462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.712108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.712364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.712632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.719695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.721139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:42.733116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:42.734158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:42.735946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 514/1052] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:42.740383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.765812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff010b-0d15-41ae-96fa-51880b8fff93 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.791852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.803644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.804325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.804518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.804782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.805441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:42.805794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.807113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.822104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 539/1053] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:42.823800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.845679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e28251-340b-4339-afe3-5fdb0b39f9d5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.881053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.881591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.881866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.882173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.882862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:42.883165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.884586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.885322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:42.896949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 515/1054] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:42.902937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:42.930621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d66b-27b9-4081-bca3-134f5183e61c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:42.967509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:42.968136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:42.968412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:42.968681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] PATH_INFO: `/v3/users/fa9d6482c2294c33bee1309269c698f0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:42.983761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] RBAC: Authorizing `identity:get_user(user_id=fa9d6482c2294c33bee1309269c698f0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:42.985251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:42.991487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 540/1055] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/users/fa9d6482c2294c33bee1309269c698f0 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:42.992472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:43.010614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474091c-180a-404b-a334-7ebee089e106 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:43.022200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:43.047970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:43.048563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:43.048837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:43.049210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:43.049965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] RBAC: Adding query filter params (role.id=c19cc115f7ef4256b96dfaf6512b4dff, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:43.050308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:43.051735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:43.057514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 516/1056] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=c19cc115f7ef4256b96dfaf6512b4dff => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:43.100943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f11b-c2be-42a4-b282-9cf16e8fff64 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:43.115488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:43.139823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:43.140356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:43.140631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:43.140896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:43.213925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:43.215433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:43.321309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 541/1057] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:43.322965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:43.429383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649f7c4-11b2-4de7-9b84-d7f987a6c343 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:43.463430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:43.464063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:43.464342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:43.464615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:43.465728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:43.467754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ffb3a-f854-4803-826d-c6eed295ef16 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:43.474429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 517/1058] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:26:43 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:43.476122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:43.491393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:43.577519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d29b6-4e94-4b59-8503-7fa64fc3bda6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:43.612116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:43.612718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:43.612991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:43.613257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:43.613965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=b91b912c0e59442cbc64f255c28ae561, user.id=fa9d6482c2294c33bee1309269c698f0) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:43.614287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:43.615686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9578c4-a6e8-49bb-9c2e-bd68925e77f2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:43.626406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 542/1059] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:43 2022] GET /identity/v3/role_assignments?user.id=fa9d6482c2294c33bee1309269c698f0&scope.project.id=b91b912c0e59442cbc64f255c28ae561&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:44.037844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcc5d0-0297-43e5-857d-fd857f13779a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:44.038439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcc5d0-0297-43e5-857d-fd857f13779a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:44.038439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcc5d0-0297-43e5-857d-fd857f13779a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:44.039941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 518/1060] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:44.044968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:44.045895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:44.046272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:44.065026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:44.066252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:44.133802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 543/1061] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:44.399295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcc5d0-0297-43e5-857d-fd857f13779a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:44.442007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:44.503775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcc5d0-0297-43e5-857d-fd857f13779a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:44.537974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:44.538602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:44.538881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:44.539152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:44.609721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:44.611240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:44.650288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:44.710095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 519/1062] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:44.715778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:44.725361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:44.816881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037004-2a57-4799-b058-b5db7fcc1591 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:44.852705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:44.853343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:44.853623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:44.853895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:44.864312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:44.865770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-694b5910-0ce5-457e-90dd-e609e61371bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:44.880402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:44.881374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:44.882303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:44.882303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:44.882303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:44.882303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 544/1063] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:44.887369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:44.995692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186599c5-8b5b-45d2-84bc-38cb031bf80a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:45.029061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:45.029725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.030004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.030302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.030941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:45.031243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:45.032604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf04c2-ace2-40c4-a500-e069492fdefa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:45.040907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:45.054085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 520/1064] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:45.226428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd377a-d695-4d31-bfe1-1309d470f80f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.226941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd377a-d695-4d31-bfe1-1309d470f80f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.227035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd377a-d695-4d31-bfe1-1309d470f80f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.228296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 545/1065] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:45.233770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.234131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.234371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.254897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:45.256020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:45.326767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 521/1066] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:45.594855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd377a-d695-4d31-bfe1-1309d470f80f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:45.655192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:45.707045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd377a-d695-4d31-bfe1-1309d470f80f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:45.740245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:45.740864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.741146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.741409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.751765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9cd7c717-9188-42d1-992d-a992af1d0ad2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:45.756515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:45.757517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:45.758431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 546/1067] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:45.763636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:45.777182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646c1f-1f38-46f0-a089-c3900dc96f85 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.777460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646c1f-1f38-46f0-a089-c3900dc96f85 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.777736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646c1f-1f38-46f0-a089-c3900dc96f85 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.779396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 547/1068] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:45.784740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.785022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.785321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.802757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:45.803953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:45.862663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819dd7c-8c6f-4360-b576-8a3addaba27d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:45.869932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 548/1069] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:45.893072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:45.898560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:45.899209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:45.899450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:45.899714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:45.906874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:45.906874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:45.906874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:45.909798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 522/1070] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:45.915709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:45.963172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.013022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffcded-e976-433d-b587-2c51aa777a68 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.046626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.047245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.047522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.047791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.057369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.058805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:46.072700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:46.073803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:46.075143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:46.075143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:46.075143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:46.078282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 549/1071] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:46.079789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.137474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.178920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbef68c-ff66-4c2e-ac87-c49847d69337 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.212035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.212631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.212910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.213181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.213816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:46.214142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.215504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.227352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 523/1072] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:46.233063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.238923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.241781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f35d9-ae8e-404e-9d90-1aa4ddb1008c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.275866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.276487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.276767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.277043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.284533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.287762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:46.293723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:46.294807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:46.295736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 550/1073] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:46.299570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.335422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d824a-98bf-469f-97ef-ca9a0e6b92ff None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.375914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.376555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.376835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.377111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.385307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.386797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:46.398832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:46.399829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:46.400811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:46.401388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1fe41-d517-4093-9e03-4bd5213f7136 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.401481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 524/1074] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:46.406307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.435158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.435792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.436065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.436333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.436993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:46.437292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.438688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.446858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 551/1075] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:46.452909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.477700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.502647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc660e-2b86-4d99-8536-bbe09bb3c2c2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.515574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.536511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.537113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.537447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.537719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.538495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:46.538797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.540268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.551226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10fd8-c7a2-4270-a0c7-565982002aa5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.557290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 525/1076] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:46.563054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.585257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.585874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.586184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.586459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] PATH_INFO: `/v3/users/fa9d6482c2294c33bee1309269c698f0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.587592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa9d6482c2294c33bee1309269c698f0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.589000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.590600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 552/1077] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/users/fa9d6482c2294c33bee1309269c698f0 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:46.597107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.665972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f603cdc-2b2f-4b56-90ac-78cd5badb4da None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.686437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.695888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184c68a-0167-4af4-b7a9-8466b8472ba1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.701341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.702579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.702859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.703131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.713965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.715748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.730516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:46.730867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:46.733181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:46.734118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:46.734118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:46.734118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:46.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.734295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.734590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 526/1078] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:46.734662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.734662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.740755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 553/1079] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:46.740900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.741703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.762467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.836915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c38c8-b245-4310-b550-41abf53cb6bb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.839091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f321b-9f6d-4966-96bf-a5bfbbab0170 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:46.869582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.870213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.870493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.870762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.871397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:46.871697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.871821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:46.872474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:46.872754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:46.873035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561/users/fa9d6482c2294c33bee1309269c698f0/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:46.873173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.875448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] RBAC: Authorizing `identity:create_grant(project_id=b91b912c0e59442cbc64f255c28ae561, user_id=fa9d6482c2294c33bee1309269c698f0, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:46.876840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:46.885245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 527/1080] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:46.890912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:46.954014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 554/1081] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:46 2022] PUT /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561/users/fa9d6482c2294c33bee1309269c698f0/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:46.970805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:46.991205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee9fd-10d9-4173-859d-36dac7f00572 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:47.035369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:47.036032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:47.036310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:47.036581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:47.044176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:47.045841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:47.058091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:47.059106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:47.060346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 528/1082] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:47.065288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:47.085923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:47.171770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f71a2c-1e6c-4ffd-87f2-9dc538363908 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:47.204878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:47.205414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:47.205691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:47.206252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:47.206943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:47.207247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:47.208657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60c049-8207-4502-a398-91fd3c513210 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:47.220410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 555/1083] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:47.226564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:47.321540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d13a4-5372-4cf1-ab9f-213e7bef327d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:47.355198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:47.355839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:47.356111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:47.356377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/c19cc115f7ef4256b96dfaf6512b4dff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:47.358738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=e7b89441837347cba3200f9104c0f16c, role_id=c19cc115f7ef4256b96dfaf6512b4dff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:47.360169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59768e95-3637-4a24-b53e-2e731cba85e8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:47.430013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 529/1084] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:47 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/e7b89441837347cba3200f9104c0f16c/roles/c19cc115f7ef4256b96dfaf6512b4dff => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:26:49.096564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453d3d7-9919-4d87-a01c-7328b855780d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.097095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453d3d7-9919-4d87-a01c-7328b855780d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.097184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453d3d7-9919-4d87-a01c-7328b855780d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.098471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 556/1085] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:49.103799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323bed0-28c8-4a97-a036-01bf17113e32 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.104211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323bed0-28c8-4a97-a036-01bf17113e32 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.104319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323bed0-28c8-4a97-a036-01bf17113e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.110539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:49.122507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3323bed0-28c8-4a97-a036-01bf17113e32 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:49.126147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3323bed0-28c8-4a97-a036-01bf17113e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:49.192612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 530/1086] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:49.459810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453d3d7-9919-4d87-a01c-7328b855780d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:49.537079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:49.557477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034eef5-a6e0-46a6-a706-dce36542648c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.557761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034eef5-a6e0-46a6-a706-dce36542648c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.558078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034eef5-a6e0-46a6-a706-dce36542648c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.559565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 531/1087] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:26:49.560722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453d3d7-9919-4d87-a01c-7328b855780d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:49.564428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.564705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.564973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.584224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:49.585391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:49.596710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:49.597330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.597607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.597874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.608463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:49.613198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:49.614393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:49.615465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 557/1088] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:26:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:49.620333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:49.654388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 532/1089] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:26:49.716959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996971-2db7-4248-9697-0b3757e014ac None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:49.750229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:49.750828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.751111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.751384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.752044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:49.752348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:49.753696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:49.758702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:49.762778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 558/1090] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:26:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:49.769290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:49.828880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:49.865605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1b9e-5394-4bd7-9b4f-4e5d0ea506b1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:49.899896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:49.900521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:49.900801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:49.901073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] PATH_INFO: `/v3/users/fa9d6482c2294c33bee1309269c698f0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:49.902218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] RBAC: Authorizing `identity:get_user(user_id=fa9d6482c2294c33bee1309269c698f0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:49.903653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:49.905292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 533/1091] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:49 2022] GET /identity/v3/users/fa9d6482c2294c33bee1309269c698f0 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:49.910952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:49.928968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.064250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ba3ca-7e39-4b1e-8025-54343cfc68ad None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.065530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacbbe7-f4d5-4e7b-8078-5eb9f818f3c3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.092602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.098190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.098328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.098815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.098908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.099072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.099176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.099336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.099599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.109774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:26:50.114611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:50.115737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:50.116767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 534/1092] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Sep 9 11:26:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:50.121977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.174606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.176901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.178433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.226959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c98a1-632c-4090-a640-909f2ca18d26 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.259716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.260319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.260603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.260876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.261538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:50.261839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.263229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.271410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 535/1093] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:50.276338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 559/1094] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:50.277190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.282203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.329315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.375051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa9409-d78d-43be-8c07-794381db36db None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.379446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae07238-5912-45d3-a068-5d75f4d46fc2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.380700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.409393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.410020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.410327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.410600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.413729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.414377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.414682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.414955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.416056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.417467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.419126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 560/1095] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:50.421163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.422631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.425163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:50.437289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:50.438450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:50.439354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:50.439354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:50.439354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:50.439354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 536/1096] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:26:50.443924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.526333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6c48b-bce3-41f1-b066-8aa61ce05976 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.543876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf5fb-6097-4f85-99c7-0916415d5302 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.546722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.560664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.561294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.561566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.561839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.562605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=b91b912c0e59442cbc64f255c28ae561, user.id=fa9d6482c2294c33bee1309269c698f0) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:50.562916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.564433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.570110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 561/1097] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/role_assignments?user.id=fa9d6482c2294c33bee1309269c698f0&scope.project.id=b91b912c0e59442cbc64f255c28ae561&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:50.577269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.578411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.579027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.579498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.580585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:50.580891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.582604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.596475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 537/1098] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:26:50.602443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.635184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:50.709448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06ff6-13e9-488f-9521-fde17bc330be None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.746650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.747280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.747559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.748640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.755730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.757156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:26:50.769297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:50.770473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:26:50.771651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 562/1099] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:50.776374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:50.876820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb4ba-9244-4abe-aad9-6bf301e8ee6d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:50.910639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:50.911245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:50.911524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:50.911796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:50.912514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:50.912817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:50.914268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:50.930952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 538/1100] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:50.936729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.025987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:51.035135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5406f24-6cb4-4cdf-9638-ffe3c736383b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:51.069067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:51.069668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:51.069944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:51.070242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:51.135073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:51.136501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:51.226927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:51.236093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 563/1101] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:26:51.242084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.267208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:51.342917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115845a-ab90-4362-9584-26e413ae41a8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:51.382367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:51.383003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:51.383339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:51.383617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:51.394207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:51.395642 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:26:51.409864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:51.410961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:26:51.411877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:26:51.411877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:26:51.411877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:51.411877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 539/1102] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:51.416887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.515471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b450c-59a5-49bc-943f-c7069afe6af6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:51.548036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:51.548635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:51.548916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:51.549188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:51.549822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:51.550149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:51.551497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:51.563800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 564/1103] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:51.569528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.641920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:51.665297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf7dbe-46c8-4c0a-8b23-d13ce91e62be None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:51.698857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:51.699472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:51.699752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:51.700025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:51.706832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:51.708271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:26:51.719362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:26:51.720367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:26:51.721289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 540/1104] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:26:51.726393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.778727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:51.825523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf25-3d64-49fc-9207-3fc38a1595ae None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:51.859000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:51.859600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:51.859877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:51.860149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:51.860833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:51.861138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:51.862580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adc8cd-86aa-4f97-8b98-b70b49b5c8db None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:51.874680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 565/1105] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:51.880944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:51.978518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf9fdd-daf8-4486-bacb-f4d86e246bd3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:52.010813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:52.011404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:52.011682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:52.011954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:52.012672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] RBAC: Adding query filter params (role.id=c19cc115f7ef4256b96dfaf6512b4dff, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=e7b89441837347cba3200f9104c0f16c) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:52.012982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:52.014483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee44de6-47a9-4002-836f-70a370bf39fe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:52.016203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:52.020546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 541/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:51 2022] GET /identity/v3/role_assignments?user.id=e7b89441837347cba3200f9104c0f16c&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=c19cc115f7ef4256b96dfaf6512b4dff => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:52.760260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654912c-5c5f-4170-9fd3-c8d154be8b53 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:52.760770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654912c-5c5f-4170-9fd3-c8d154be8b53 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:52.760770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654912c-5c5f-4170-9fd3-c8d154be8b53 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:52.762098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 566/1107] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:52.768499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:52.768938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:52.769052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:52.787147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:52.788334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:52.856113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 542/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:53.123077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654912c-5c5f-4170-9fd3-c8d154be8b53 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:53.222466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654912c-5c5f-4170-9fd3-c8d154be8b53 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:53.234759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:53.254175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:53.254782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:53.255055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:53.255322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:53.256436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:53.257837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea5a94-1e2a-4d07-be7b-b181f1c8b44d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:53.259399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 567/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:26:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:53.266106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:53.361034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7b820-ff0f-4c8a-8f41-fe11310b623d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:53.386894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:53.395044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:53.395645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:53.395924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:53.397665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:53.397665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:53.398331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b61fc0-6cfc-446b-ac14-0153b22ae0fb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:53.504201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 543/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:26:53 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:26:55.680423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da43b9-44c1-4877-ad21-9d248b9329c0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:55.680922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da43b9-44c1-4877-ad21-9d248b9329c0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:55.680922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da43b9-44c1-4877-ad21-9d248b9329c0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:55.682252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 568/1111] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:55.687948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c022-32eb-4865-971f-a435731a797b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:55.688373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c022-32eb-4865-971f-a435731a797b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:55.688490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c022-32eb-4865-971f-a435731a797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:55.707308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d345c022-32eb-4865-971f-a435731a797b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:55.708487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d345c022-32eb-4865-971f-a435731a797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:55.773850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 544/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:56.046759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da43b9-44c1-4877-ad21-9d248b9329c0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:56.144838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da43b9-44c1-4877-ad21-9d248b9329c0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:56.177190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:56.177797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:56.178096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:56.178376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] PATH_INFO: `/v3/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:56.179760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] RBAC: Authorizing `identity:get_role(role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:56.182926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:56.182926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 569/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:56 2022] GET /identity/v3/roles/28d83235f01b49b98d223883c8317b87 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:56.188134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c022-32eb-4865-971f-a435731a797b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:56.284954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c022-32eb-4865-971f-a435731a797b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:56.298349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:56.317098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:56.317716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:56.317992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:56.318291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] PATH_INFO: `/v3/users/f53a2ed62ac948f1b83cedaa28d98a49` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:56.332102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] RBAC: Authorizing `identity:get_user(user_id=f53a2ed62ac948f1b83cedaa28d98a49)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:56.333529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:56.335139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 545/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:56 2022] GET /identity/v3/users/f53a2ed62ac948f1b83cedaa28d98a49 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:56.341257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:56.392957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:56.441520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981ce46-9a95-4486-8570-2410c7c20714 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:56.474379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:56.474975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:56.475255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:56.475527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:56.539119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:56.540540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:56.586412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:56.635291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 570/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:26:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:26:56.641935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:56.739591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76202e51-5add-45bf-ba1f-5cebae876b21 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:56.773657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:56.774296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:56.774574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:56.774843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:56.775949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:56.777368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb857b8-6cef-41fe-a361-69ee3aa3d394 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:56.779007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 546/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:26:56 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:56.785234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:56.881085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434484-5997-4813-ab3d-44ce60844c84 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:56.912966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:56.913552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:56.913829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:56.914128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:56.914841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b91b912c0e59442cbc64f255c28ae561, user.id=f53a2ed62ac948f1b83cedaa28d98a49) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:26:56.915147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:56.916538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d812e-2503-45f7-9908-cb31b74ee85c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:56.922210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 571/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:26:56 2022] GET /identity/v3/role_assignments?user.id=f53a2ed62ac948f1b83cedaa28d98a49&scope.project.id=b91b912c0e59442cbc64f255c28ae561&role.id=28d83235f01b49b98d223883c8317b87 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:59.028907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112fb0e-c7e4-4bf5-b5b4-6b69af0e9f5d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.029437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112fb0e-c7e4-4bf5-b5b4-6b69af0e9f5d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.029437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112fb0e-c7e4-4bf5-b5b4-6b69af0e9f5d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.031355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 547/1118] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:26:59.036637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.037059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.037169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.054814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:26:59.055999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:26:59.063117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:59.120871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 572/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:26:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:26:59.392374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112fb0e-c7e4-4bf5-b5b4-6b69af0e9f5d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:59.433214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:59.489640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112fb0e-c7e4-4bf5-b5b4-6b69af0e9f5d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:59.522604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:59.523222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.523495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.523762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] PATH_INFO: `/v3/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.525131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] RBAC: Authorizing `identity:get_role(role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:59.526528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:59.527839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 548/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:59 2022] GET /identity/v3/roles/28d83235f01b49b98d223883c8317b87 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:59.534575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:59.630208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972155-17b3-4c4d-a5ec-6ebcade43daf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:59.662494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:59.663111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.663391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.663664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] PATH_INFO: `/v3/users/f53a2ed62ac948f1b83cedaa28d98a49` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.664778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] RBAC: Authorizing `identity:get_user(user_id=f53a2ed62ac948f1b83cedaa28d98a49)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:59.666219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:59.667777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 573/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:26:59 2022] GET /identity/v3/users/f53a2ed62ac948f1b83cedaa28d98a49 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:59.674167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:59.769811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca124-b36d-43e8-8b82-1c9229c93084 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:59.778679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:59.803688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:59.804102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.804220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.804427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.805556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:59.807039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169052b-cede-4890-aad9-3d5bbb28ccdd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:26:59.808672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 549/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:26:59 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:26:59.814914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:26:59.826863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:26:59.910798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd1c42-a9a7-4cbe-8186-1366347e4fca None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:26:59.942517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:26:59.943158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:26:59.943433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:26:59.943704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561/users/f53a2ed62ac948f1b83cedaa28d98a49/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:26:59.945799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b91b912c0e59442cbc64f255c28ae561, user_id=f53a2ed62ac948f1b83cedaa28d98a49, role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:26:59.947226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dd2c1-8be2-4b89-b63c-331d8346aef5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:00.006188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 574/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:26:59 2022] PUT /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561/users/f53a2ed62ac948f1b83cedaa28d98a49/roles/28d83235f01b49b98d223883c8317b87 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:27:02.130492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872205-eaee-45a7-9665-c4eda0b01b7d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:02.131025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872205-eaee-45a7-9665-c4eda0b01b7d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:02.131025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09872205-eaee-45a7-9665-c4eda0b01b7d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:02.132411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 550/1124] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:02.137775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:02.138231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:02.138339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:02.156245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:02.157430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:02.217508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:02.222185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 575/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:02.489683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09872205-eaee-45a7-9665-c4eda0b01b7d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:02.587962 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09872205-eaee-45a7-9665-c4eda0b01b7d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:02.619696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:02.620304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:02.620581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:02.620854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] PATH_INFO: `/v3/roles/28d83235f01b49b98d223883c8317b87` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:02.622247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] RBAC: Authorizing `identity:get_role(role_id=28d83235f01b49b98d223883c8317b87)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:02.623606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:02.625435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 551/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:02 2022] GET /identity/v3/roles/28d83235f01b49b98d223883c8317b87 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:02.631163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:02.724918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcf917-0b3e-4491-bec9-95e19c2ff083 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:02.760156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:02.760718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:02.760996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:02.761267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] PATH_INFO: `/v3/users/f53a2ed62ac948f1b83cedaa28d98a49` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:02.762410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=f53a2ed62ac948f1b83cedaa28d98a49)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:02.763820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:02.765424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 576/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:02 2022] GET /identity/v3/users/f53a2ed62ac948f1b83cedaa28d98a49 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:02.771088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:02.817964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:02.870681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f951ff-700a-4454-8c2f-36627507e10b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:02.903893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:02.904492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:02.904770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:02.905044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:02.969878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:02.971342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:03.013512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:03.065575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 552/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:27:03.071553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:03.137300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:03.168454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68d396-0603-40e8-b111-7f520f9dfcb1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:03.201657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:03.202361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:03.202637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:03.202914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] PATH_INFO: `/v3/projects/b91b912c0e59442cbc64f255c28ae561` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:03.204021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] RBAC: Authorizing `identity:get_project(project_id=b91b912c0e59442cbc64f255c28ae561)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:03.205436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd0aa2-39b0-46dc-8736-df8b91ec08b7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:03.207061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 577/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:03 2022] GET /identity/v3/projects/b91b912c0e59442cbc64f255c28ae561 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:03.213365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:03.309477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d0d15-ee95-4a20-ac9d-c2396e4c8e49 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:03.341498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:03.342109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:03.342388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:03.342658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:03.343902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] RBAC: Adding query filter params (role.id=28d83235f01b49b98d223883c8317b87, scope.project.id=b91b912c0e59442cbc64f255c28ae561, user.id=f53a2ed62ac948f1b83cedaa28d98a49) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:03.344244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:03.345686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fefeac-9642-4965-8272-4c465d0e9dbe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:03.351426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 553/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:27:03 2022] GET /identity/v3/role_assignments?user.id=f53a2ed62ac948f1b83cedaa28d98a49&scope.project.id=b91b912c0e59442cbc64f255c28ae561&role.id=28d83235f01b49b98d223883c8317b87 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:05.512103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c72d4-ab68-4dd8-8dbd-8a43b9e282f1 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:05.512661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c72d4-ab68-4dd8-8dbd-8a43b9e282f1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:05.512661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c72d4-ab68-4dd8-8dbd-8a43b9e282f1 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:05.513967 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 578/1131] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:05.519919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:05.520340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:05.520452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:05.538205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:05.539389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:05.541293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:05.603851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 554/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:27:05.872430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c72d4-ab68-4dd8-8dbd-8a43b9e282f1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:05.969063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c72d4-ab68-4dd8-8dbd-8a43b9e282f1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:06.001046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:06.001647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:06.001949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:06.002249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:06.003346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:06.004749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40079f-b9e8-447f-ace1-0d11b2b20ec0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:06.006283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 579/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:06.011916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:06.106809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771b657-e526-4093-a3b3-7f7f2d4fe048 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:06.139101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:06.139718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:06.140054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:06.140327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:06.140963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:06.142406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737dc37-2840-48ee-91b9-785f61a430da None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:06.151484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:06.247047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 555/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:27:06 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:27:08.408536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09361bec-5a53-4e78-bcd4-f1a65dd89552 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:08.408957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09361bec-5a53-4e78-bcd4-f1a65dd89552 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:08.409073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09361bec-5a53-4e78-bcd4-f1a65dd89552 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:08.410441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 580/1135] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:08.415694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:08.416192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:08.416289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:08.435679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:08.437434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:08.500820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 556/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:08.767456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09361bec-5a53-4e78-bcd4-f1a65dd89552 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:08.771223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:08.866795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09361bec-5a53-4e78-bcd4-f1a65dd89552 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:08.904554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:08.905171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:08.905451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:08.905845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:08.907006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:08.908425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256eea8-f528-4a93-9e78-e05ea462b583 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:08.909988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 581/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:08.915749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:09.010516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49605-c645-41ce-8a40-0ef9ddc266f3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:09.029497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:09.042624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:09.043209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:09.043484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:09.043752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:09.044400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:09.045860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6a97-e49b-4bd0-b735-9ac8f25ff23a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:09.132914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 557/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:27:08 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:27:11.289698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a517-bc51-4ad6-a09a-bad137e23470 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:11.290253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a517-bc51-4ad6-a09a-bad137e23470 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:11.290253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a517-bc51-4ad6-a09a-bad137e23470 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:11.291636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 582/1139] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:11.297284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:11.297699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:11.297830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:11.316087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:11.317267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:11.382265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 558/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:11.650566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a517-bc51-4ad6-a09a-bad137e23470 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:11.720876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:11.749533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a517-bc51-4ad6-a09a-bad137e23470 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:11.783469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:11.784017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:11.784295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:11.784566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:11.794239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:11.798998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:11.800113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:11.801209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 583/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:11.806184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:11.900977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c14b-fb9e-48f0-8b26-299c1e8e041d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:11.908442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:11.933871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:11.934541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:11.935147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:11.935464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:11.936125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:11.936428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:11.937831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:11.945453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 559/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:11.951447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:12.056968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ab6f7-1d2b-47a9-b700-94f94780bdb7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:12.085787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:12.090459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:12.091069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:12.091347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:12.091620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] PATH_INFO: `/v3/users/671b69efb0ad41368f4e8a7ca3deed63` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:12.104733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] RBAC: Authorizing `identity:get_user(user_id=671b69efb0ad41368f4e8a7ca3deed63)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:12.106200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:12.107764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 584/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:11 2022] GET /identity/v3/users/671b69efb0ad41368f4e8a7ca3deed63 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:12.113578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:12.212928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b984571-6ba6-4ed2-a444-a523c7605e47 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:12.244833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:12.245423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:12.245698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:12.245971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:12.279154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:12.309658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:12.311113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:12.406636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 560/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:27:12.412475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:12.508700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e83037-4ab7-4c02-a9f4-a7bbe65049c3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:12.540331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:12.540946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:12.541222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:12.541491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] PATH_INFO: `/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:12.542622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] RBAC: Authorizing `identity:get_project(project_id=701b5b3832d84e98b6b86aca8c38bfa9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:12.544033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23a68-14ed-4f28-be4e-f20cd61ac60a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:12.545613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 585/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:12 2022] GET /identity/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:12.552213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:12.616232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:12.647413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55bce1-e67b-4aa4-b007-fae5f96cd79e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:12.679832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:12.680422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:12.680697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:12.680964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:12.681677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=701b5b3832d84e98b6b86aca8c38bfa9, user.id=671b69efb0ad41368f4e8a7ca3deed63) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:12.681982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:12.683420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53420b-ad54-4a48-9c05-4d426860d868 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:12.689124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 561/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:27:12 2022] GET /identity/v3/role_assignments?user.id=671b69efb0ad41368f4e8a7ca3deed63&scope.project.id=701b5b3832d84e98b6b86aca8c38bfa9&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:14.782353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38deb0-63de-43d9-a01a-fde0125abfb3 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:14.782913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38deb0-63de-43d9-a01a-fde0125abfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:14.782913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38deb0-63de-43d9-a01a-fde0125abfb3 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:14.784144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 586/1147] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:14.789359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:14.789799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:14.789927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:14.808078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:14.809253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:14.873482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 562/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:15.148097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38deb0-63de-43d9-a01a-fde0125abfb3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:15.210365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:15.251229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38deb0-63de-43d9-a01a-fde0125abfb3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:15.283730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:15.284345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:15.284622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:15.284892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:15.294624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:15.300038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:15.301239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:15.302420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 587/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:15.307374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:15.366738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:15.403376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01bf42-471f-4622-98e3-0132b71e6432 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:15.437204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:15.437802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:15.438110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:15.438378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:15.439037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:15.439338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:15.440694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:15.448396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 563/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:15.454855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:15.551674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2ce0-32ee-4d23-a791-55dabaf40e38 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:15.568222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:15.584808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:15.585420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:15.585699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:15.585972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] PATH_INFO: `/v3/users/671b69efb0ad41368f4e8a7ca3deed63` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:15.587117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] RBAC: Authorizing `identity:get_user(user_id=671b69efb0ad41368f4e8a7ca3deed63)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:15.588534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:15.590124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 588/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:15 2022] GET /identity/v3/users/671b69efb0ad41368f4e8a7ca3deed63 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:15.596561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:15.691926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1378-3840-437a-9e07-4a50f3c34ae5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:15.714228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:15.724861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:15.725477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:15.725757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:15.726030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] PATH_INFO: `/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:15.727166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] RBAC: Authorizing `identity:get_project(project_id=701b5b3832d84e98b6b86aca8c38bfa9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:15.728594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb14b7-2258-4895-9bbf-1a83dfc78b15 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:15.730231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 564/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:15 2022] GET /identity/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:15.736145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:15.832068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288ed6-3e60-48e7-8719-f9ebda0b3223 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:15.864690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:15.865322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:15.865594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:15.865864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] PATH_INFO: `/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9/users/671b69efb0ad41368f4e8a7ca3deed63/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:15.867747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] RBAC: Authorizing `identity:create_grant(project_id=701b5b3832d84e98b6b86aca8c38bfa9, user_id=671b69efb0ad41368f4e8a7ca3deed63, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:15.869151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591edbf-27e4-4ac0-9a14-83379356e226 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:15.939256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 589/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:27:15 2022] PUT /identity/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9/users/671b69efb0ad41368f4e8a7ca3deed63/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:27:18.056423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd53b6-4e33-45b8-8873-dd73d6ed9d7b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.056979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd53b6-4e33-45b8-8873-dd73d6ed9d7b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:18.056979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd53b6-4e33-45b8-8873-dd73d6ed9d7b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:18.058339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 565/1154] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:18.063670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.064100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:18.064201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:18.082321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:18.083750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:18.123916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:18.148181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 590/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:18.426046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd53b6-4e33-45b8-8873-dd73d6ed9d7b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:18.524785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd53b6-4e33-45b8-8873-dd73d6ed9d7b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:18.557244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:18.557858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.558163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:18.558439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:18.569385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:18.574181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:18.575286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:27:18.576333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 566/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:18.581380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:18.677309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcefda-d4cc-4635-83c8-68301dd5daa1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:18.710998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:18.711588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.711865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:18.712139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:18.712795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:18.713095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:18.714475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:18.722208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 591/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:18.728095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:18.779444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:18.824997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8ea-d61c-4f25-9538-62baf4b1746e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:18.857975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:18.858647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.858957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:18.859784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] PATH_INFO: `/v3/users/671b69efb0ad41368f4e8a7ca3deed63` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:18.860883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=671b69efb0ad41368f4e8a7ca3deed63)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:18.862329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:18.864537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 567/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:18 2022] GET /identity/v3/users/671b69efb0ad41368f4e8a7ca3deed63 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:18.869782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:18.908787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:18.966131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2010b-a9fe-4b9c-8f7b-27aea7665821 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:18.998600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:18.999194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:18.999472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:19.000239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:19.068992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:19.070458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:19.106961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:19.164332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 592/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:27:19.170407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:19.265526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019065c-4dca-447c-8a93-6e918a5f0ef0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:19.283721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:19.297286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:19.297896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:19.298290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:19.298563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] PATH_INFO: `/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:19.299659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] RBAC: Authorizing `identity:get_project(project_id=701b5b3832d84e98b6b86aca8c38bfa9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:19.301070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429aeec-21e8-4dbe-9763-2d852f8adbfc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:19.302689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 568/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:19 2022] GET /identity/v3/projects/701b5b3832d84e98b6b86aca8c38bfa9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:19.309050 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:19.405492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868a01-b18c-4017-8d57-c87bd560d807 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:19.438864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:19.439462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:19.439739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:19.440011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:19.440730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=701b5b3832d84e98b6b86aca8c38bfa9, user.id=671b69efb0ad41368f4e8a7ca3deed63) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:19.441037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:19.442482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180de81-ef09-4a12-8034-525f61ea60e0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:19.448021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 593/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:27:19 2022] GET /identity/v3/role_assignments?user.id=671b69efb0ad41368f4e8a7ca3deed63&scope.project.id=701b5b3832d84e98b6b86aca8c38bfa9&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:21.593224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762820-4449-40b6-9ac0-fe56b5ff2a36 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:21.593814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762820-4449-40b6-9ac0-fe56b5ff2a36 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:21.593814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e762820-4449-40b6-9ac0-fe56b5ff2a36 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:21.595191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 569/1162] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:21.600407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e40a78-7add-4714-9643-e25a72096919 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:21.600824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e40a78-7add-4714-9643-e25a72096919 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:21.600933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e40a78-7add-4714-9643-e25a72096919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:21.607237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:21.619373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-60e40a78-7add-4714-9643-e25a72096919 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:21.620530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-60e40a78-7add-4714-9643-e25a72096919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:21.684340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 594/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:21.950742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762820-4449-40b6-9ac0-fe56b5ff2a36 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:22.051406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e762820-4449-40b6-9ac0-fe56b5ff2a36 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:22.084283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:22.084871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:22.085146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:22.085414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:22.136980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:22.149421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:22.150878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:22.244127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 570/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:27:22.250595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e40a78-7add-4714-9643-e25a72096919 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:22.344312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e40a78-7add-4714-9643-e25a72096919 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:22.372364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:22.377737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:22.378411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:22.378693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:22.379391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:22.385916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:22.387385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3725fe2e-1ea8-479a-9717-93073cc5f1f3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:22.399407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:22.401317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:22.402469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 595/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:27:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:22.406535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:22.493798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:22.508753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bed02-769b-42c6-b5d6-f3db6348ff7d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:22.537648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:22.538264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:22.538546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:22.538818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:22.539454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:22.539759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:22.541115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd7cf8-9665-4ff3-9631-42f0d8e3cb89 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:22.548983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 571/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Sep 9 11:27:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:24.668360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7fc5-0a58-43fa-8125-29dcbac3b4f7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:24.668862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7fc5-0a58-43fa-8125-29dcbac3b4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:24.668862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7fc5-0a58-43fa-8125-29dcbac3b4f7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:24.670151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 596/1167] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:24.676187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a637-9b29-4041-83e8-c177f23cd698 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:24.676609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a637-9b29-4041-83e8-c177f23cd698 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:24.676715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a637-9b29-4041-83e8-c177f23cd698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:24.695105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e1b3a637-9b29-4041-83e8-c177f23cd698 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:24.696286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e1b3a637-9b29-4041-83e8-c177f23cd698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:24.759636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 572/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:25.027476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7fc5-0a58-43fa-8125-29dcbac3b4f7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:25.128768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7fc5-0a58-43fa-8125-29dcbac3b4f7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:25.161132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:25.161722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:25.162001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:25.162300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:25.162973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:25.164316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6e807-5e3b-4f78-b18f-405648fea26b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:25.231469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 597/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Fri Sep 9 11:27:25 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:27:27.401010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395dc63-4e61-4989-8659-7269059b87f0 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:27.401517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395dc63-4e61-4989-8659-7269059b87f0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:27.401616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395dc63-4e61-4989-8659-7269059b87f0 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:27.402941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 573/1170] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:27.408263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:27.408672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:27.408813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:27.426572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:27.427752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:27.473574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:27.492781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 598/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:27.767044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395dc63-4e61-4989-8659-7269059b87f0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:27.848238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:27.866922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395dc63-4e61-4989-8659-7269059b87f0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:27.899225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:27.899837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:27.900115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:27.900384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] PATH_INFO: `/v3/domains/60ef2342cb8344e08253d1efb1b69e76` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:27.910932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] RBAC: Authorizing `identity:get_domain(domain_id=60ef2342cb8344e08253d1efb1b69e76)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:27.912412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb5610-1930-4865-9c33-30a4fe1487be None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:27.914126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 574/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Fri Sep 9 11:27:27 2022] GET /identity/v3/domains/60ef2342cb8344e08253d1efb1b69e76 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:27.920080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:28.017002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0f8b0-fa8c-41fd-885d-6c7ff2d41ce8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:28.050033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:28.050599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:28.050877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:28.051149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:28.051792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:28.053204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e87009-319f-42c3-b446-a102ddee2cec None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:28.123293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 599/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:27:27 2022] POST /identity/v3/projects => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:27:30.256922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0dde-cb14-44e2-be22-7e54264db853 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:30.257453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0dde-cb14-44e2-be22-7e54264db853 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:30.257453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0dde-cb14-44e2-be22-7e54264db853 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:30.258836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 575/1174] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:30.263914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:30.264284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:30.264455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:30.283517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:30.285524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:30.313445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:30.352641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 600/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:30.624604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e0dde-cb14-44e2-be22-7e54264db853 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:30.721643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e0dde-cb14-44e2-be22-7e54264db853 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:30.754246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:30.754845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:30.757036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:30.757036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] PATH_INFO: `/v3/domains/60ef2342cb8344e08253d1efb1b69e76` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:30.757036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=60ef2342cb8344e08253d1efb1b69e76)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:30.757982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf064934-937f-4886-800e-19eb700dc689 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:30.759539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 576/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Fri Sep 9 11:27:30 2022] GET /identity/v3/domains/60ef2342cb8344e08253d1efb1b69e76 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:30.765186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:30.862429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5d74-71db-4386-9329-f7c7c2eda04f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:30.894671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:30.895210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:30.895486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:30.895757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:30.896411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:30.897976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54283a-cc51-4fa1-802f-da589b0c8d40 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:31.007314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 601/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:27:30 2022] POST /identity/v3/users => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:27:33.172551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaa22e-21e9-41ce-b01c-bf970e134c93 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:33.173112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaa22e-21e9-41ce-b01c-bf970e134c93 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:33.173112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaa22e-21e9-41ce-b01c-bf970e134c93 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:33.174462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 577/1178] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:33.179610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:33.180032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:33.180141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:33.198544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:33.199715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:33.213384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:33.264478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 602/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:33.532271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaa22e-21e9-41ce-b01c-bf970e134c93 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:33.545924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:33.639592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaa22e-21e9-41ce-b01c-bf970e134c93 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:33.674720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:33.675346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:33.675622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:33.675892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:33.686211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:33.691044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:33.692040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:33.692909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:27:33.698103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 578/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:27:33.698249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:33.798957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d0746-602b-45c3-a4cb-cff3a74161c9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:33.832217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:33.832820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:33.833100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:33.833379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:33.834301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:33.834632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:33.836043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:33.844562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 603/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:33.850584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:33.917512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:33.948390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f7de4-3c65-41a6-b535-446d10967440 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:33.982851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:33.983512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:33.983793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:33.984068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] PATH_INFO: `/v3/users/ff790a41bc1846068a20f893221139ff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:33.997636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] RBAC: Authorizing `identity:get_user(user_id=ff790a41bc1846068a20f893221139ff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:33.999157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:34.000746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 579/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:33 2022] GET /identity/v3/users/ff790a41bc1846068a20f893221139ff => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:34.006682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:34.021660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:34.105099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26cf90-1cd8-4871-8440-b2de926b98e8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:34.137859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:34.138479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:34.138760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:34.139032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:34.203597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:34.205014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:34.217957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:34.299593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 604/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:27:34.305478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:34.400856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd089d-ab87-4152-81d7-4777937eae59 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:34.426066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:34.433415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:34.434034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:34.434338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:34.434659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] PATH_INFO: `/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:34.435763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] RBAC: Authorizing `identity:get_project(project_id=60bde8f64db74b9ba9f7d7425e16f1c5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:34.437180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b423f-0f88-47bd-a502-1293efb93b16 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:34.438817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 580/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:34 2022] GET /identity/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:34.444975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:34.541135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a4e97-a589-42f1-8e25-10e0e7a63988 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:34.553544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:34.573537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:34.574145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:34.574420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:34.574682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:34.575385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=60bde8f64db74b9ba9f7d7425e16f1c5, user.id=ff790a41bc1846068a20f893221139ff) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:34.575685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:34.577072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117137c-2e8c-4a7e-8a76-af8465918fcb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:34.582157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 605/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:27:34 2022] GET /identity/v3/role_assignments?user.id=ff790a41bc1846068a20f893221139ff&scope.project.id=60bde8f64db74b9ba9f7d7425e16f1c5&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:36.693762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3a930-68b6-43e5-84fe-2af2cc2a9992 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:36.694401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3a930-68b6-43e5-84fe-2af2cc2a9992 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:36.694401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3a930-68b6-43e5-84fe-2af2cc2a9992 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:36.695591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 581/1186] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:36.700769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:36.701199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:36.701299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:36.720519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:36.721703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:36.786406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 606/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:37.053048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3a930-68b6-43e5-84fe-2af2cc2a9992 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:37.151128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3a930-68b6-43e5-84fe-2af2cc2a9992 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:37.183791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:37.184400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:37.184679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:37.184950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:37.197557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:37.202384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:37.204152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:27:37.205020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 582/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:37.210572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:37.304461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af116-373f-4807-923b-0241ec6cd2d2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:37.311319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:37.337359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:37.337939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:37.338248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:37.338519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:37.339173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:37.339473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:37.340807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:37.349619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 607/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:37.356145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:37.413822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:37.456221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0acd5-b02b-4325-b57f-d97241a5120b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:37.490219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:37.490828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:37.491105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:37.491376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] PATH_INFO: `/v3/users/ff790a41bc1846068a20f893221139ff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:37.492490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ff790a41bc1846068a20f893221139ff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:37.493913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:37.495513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 583/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:37 2022] GET /identity/v3/users/ff790a41bc1846068a20f893221139ff => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:37.501250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:37.597155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2984e-0956-4f37-a227-119cd5df41f3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:37.628846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:37.629453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:37.629729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:37.630001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] PATH_INFO: `/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:37.631136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] RBAC: Authorizing `identity:get_project(project_id=60bde8f64db74b9ba9f7d7425e16f1c5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:37.632539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dd651-b837-4c76-9974-7c78306892ca None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:37.634142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 608/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:37 2022] GET /identity/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:37.639865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:37.734951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df771d-54a8-4382-a12e-c65ab97411e6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:37.757360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:37.768035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:37.768675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:37.768955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:37.769226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] PATH_INFO: `/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5/users/ff790a41bc1846068a20f893221139ff/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:37.771115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=60bde8f64db74b9ba9f7d7425e16f1c5, user_id=ff790a41bc1846068a20f893221139ff, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:37.772547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ff991-3315-43f6-821c-90a3f32a36e8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:37.840938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 584/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:27:37 2022] PUT /identity/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5/users/ff790a41bc1846068a20f893221139ff/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:27:39.955409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2423e6-6a8f-43c3-be30-ecb58bfd3c35 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:39.955939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2423e6-6a8f-43c3-be30-ecb58bfd3c35 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:39.955939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2423e6-6a8f-43c3-be30-ecb58bfd3c35 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:39.957298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 609/1193] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:27:39.962792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:39.963198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:39.963341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:39.981592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:27:39.982812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:27:40.048343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 585/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:27:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:27:40.316535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2423e6-6a8f-43c3-be30-ecb58bfd3c35 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:40.352694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:40.416135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2423e6-6a8f-43c3-be30-ecb58bfd3c35 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:40.447781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:40.448392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:40.448665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:40.448932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:40.459104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:27:40.464108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:27:40.465281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:27:40.466161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 610/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:27:40.471707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:40.566229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d1f05-6669-418f-97b5-c757f09d2b2c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:40.579986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:40.599802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:40.600389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:40.600776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:40.601048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:40.601703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:40.602001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:40.603386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:40.611486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 586/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:27:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:40.617304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:40.707654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:40.713369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b65cb-9575-4b8f-86de-29ad6959227e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:40.746922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:40.747534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:40.747812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:40.748084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] PATH_INFO: `/v3/users/ff790a41bc1846068a20f893221139ff` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:40.749192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] RBAC: Authorizing `identity:get_user(user_id=ff790a41bc1846068a20f893221139ff)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:40.750682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:40.752238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 611/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Sep 9 11:27:40 2022] GET /identity/v3/users/ff790a41bc1846068a20f893221139ff => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:40.757922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:40.854699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaf88b-0c25-473b-a9fc-f6e99f9f5358 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:40.887025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:40.887617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:40.887897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:40.888170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:40.931623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:40.953689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:40.955153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:41.049304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 587/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:27:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:27:41.055221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:41.150787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49406a-f88d-4327-b9cc-70dea8df077a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:41.183897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:41.184501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:41.184778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:41.185049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] PATH_INFO: `/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:41.186182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] RBAC: Authorizing `identity:get_project(project_id=60bde8f64db74b9ba9f7d7425e16f1c5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:41.187660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d4e5d-fc7e-42f9-9e14-0589751b1066 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:41.189254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 612/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Sep 9 11:27:41 2022] GET /identity/v3/projects/60bde8f64db74b9ba9f7d7425e16f1c5 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:41.195249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:27:41.272590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:41.294856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7142946-6901-4e49-8d58-a58eb97e8b5a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:27:41.327369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:27:41.328479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:27:41.328754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:27:41.329025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:27:41.329737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=60bde8f64db74b9ba9f7d7425e16f1c5, user.id=ff790a41bc1846068a20f893221139ff) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:27:41.330042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:27:41.331488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ef65c-6ff2-46c9-94e0-5d8b2eece7dc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:27:41.337260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 588/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:27:41 2022] GET /identity/v3/role_assignments?user.id=ff790a41bc1846068a20f893221139ff&scope.project.id=60bde8f64db74b9ba9f7d7425e16f1c5&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:27:51.179857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=84 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:27:51.334545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=30 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:28:57.557534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5f7b8-f2d5-4db1-8746-ff880ae17947 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:28:57.558139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5f7b8-f2d5-4db1-8746-ff880ae17947 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:28:57.558139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5f7b8-f2d5-4db1-8746-ff880ae17947 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:28:57.559430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 613/1201] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:28:57.564655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:28:57.565114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:28:57.565265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:28:57.583918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:28:57.585104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:28:57.650448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 589/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:28:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:28:57.920342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5f7b8-f2d5-4db1-8746-ff880ae17947 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:28:58.019582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5f7b8-f2d5-4db1-8746-ff880ae17947 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:28:58.051794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:28:58.052406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:28:58.052684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:28:58.052957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:28:58.063455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:28:58.064920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:28:58.079911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:28:58.080994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:28:58.082032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:28:58.082032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:28:58.082032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:28:58.082032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 614/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:28:58.087426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:28:58.188877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ba9f-ba0d-4a9e-947e-8ecae7faee90 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:28:58.217494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:28:58.222853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:28:58.223480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:28:58.223811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:28:58.224131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:28:58.224715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:28:58.225009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:28:58.226418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a355c-b9ca-4998-9ae1-e79230f0d8d2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:28:58.237735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 590/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:28:58.243537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:28:58.295174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:28:58.340144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba2ded-c7dc-4e74-a951-700576de260d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:28:58.374908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:28:58.375511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:28:58.375790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:28:58.376057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:28:58.376710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:28:58.380666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f3f0-3b9d-4202-a6f2-199951341e8a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:28:58.533324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 615/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Sep 9 11:28:58 2022] POST /identity/v3/users => generated 251 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:29:00.696971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2a0d7-f123-4806-9a9f-4071439e037f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:00.697474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2a0d7-f123-4806-9a9f-4071439e037f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:00.697474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2a0d7-f123-4806-9a9f-4071439e037f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:00.698839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 591/1206] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:00.704495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:00.704888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:00.705047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:00.723334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:00.724510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:00.788979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 616/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:01.061628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2a0d7-f123-4806-9a9f-4071439e037f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.161863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2a0d7-f123-4806-9a9f-4071439e037f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.195340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.195958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.196235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.196510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.207330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:01.212320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:01.213296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:01.214321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 592/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:01.220319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.258285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:01.315902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c664b33-ac2f-405d-bf00-31db920a7573 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.349404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.350002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.350309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.350582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.351242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:01.351545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:01.352894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:01.360428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 617/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:01.366391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.418045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:01.466211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbef91-7fa3-4ace-a394-8781fb192a74 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.498692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.499313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.499587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.499856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.510277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:01.511720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:01.525838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:01.527066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:01.528198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:01.528198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:01.528198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:01.528198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 593/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:01.533585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.627641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:01.630350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b7aa-3b81-4b06-b184-540707bbceb5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.663479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.664077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.664354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.664624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.665265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:01.665565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:01.666951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:01.677671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 618/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:01.683944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.782775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a0ff1-91f0-46a3-b4da-29165b5adeaa None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.784310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:01.815954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.816578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.816878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.817147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.824615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:01.826737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:01.838101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:01.843878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:01.845131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 594/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:01.845903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:01.943511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc359-6639-4353-b406-3bf3a0d20dba None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:01.976376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:01.978349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:01.978349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:01.978349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:01.978349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:01.978714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:01.980013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:01.997135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 619/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:02.003628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:02.100946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8efb53-6d7c-41bb-8e67-27a07829d129 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:02.134336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:02.134929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:02.135206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:02.135476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:02.163852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:02.200913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:02.202462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:02.298315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 595/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:29:02.304432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:02.323738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:02.404629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01290b22-b258-49da-8daf-1f32508af1a1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:02.437967 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:02.438696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:02.438974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:02.439246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:02.449020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:02.450504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:02.465132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:02.466110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:02.473407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:02.473407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:02.473407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:02.473407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 620/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:02.473688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:02.532288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:02.571148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe046382-cf22-48b3-9dca-5e1e77184dc6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:02.605388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:02.605981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:02.606288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:02.606560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:02.607774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:02.608077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:02.609443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:02.620430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 596/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:02.626237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:02.693788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:02.724854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165fd5e-f0ec-42a4-86fb-484f1edf0de5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:02.756290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:02.759934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:02.759934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:02.759934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:02.764383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:02.765833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:02.776898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:02.777975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:02.779469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 621/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:02.784558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:02.882128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2b48-f713-42ef-b593-435a3bcb59db None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:02.905255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:02.916530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:02.917136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:02.917654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:02.917925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:02.918635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:02.918938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:02.920391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad592c-da62-4011-9ae5-882693fc6af0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:02.932681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 597/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:02.938911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:03.036419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad589d-6bab-4a1b-bbe0-5e539336a995 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:03.063498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:03.069153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:03.069745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:03.070024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:03.070323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:03.071039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=fa0c95e5281f4f9d9dac910415d10932) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:03.071346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:03.072753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52451c-5ad4-4070-9001-e455b947b525 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:03.078574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 622/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:29:02 2022] GET /identity/v3/role_assignments?user.id=fa0c95e5281f4f9d9dac910415d10932&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:05.193557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cfce0-e39d-46e1-8d97-e7bc9ec5026b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:05.194166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cfce0-e39d-46e1-8d97-e7bc9ec5026b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:05.194166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cfce0-e39d-46e1-8d97-e7bc9ec5026b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:05.195475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 598/1220] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:29:05.200483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:05.201413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:05.201557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:05.224307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:05.225604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:05.290816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 623/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:05.563430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cfce0-e39d-46e1-8d97-e7bc9ec5026b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:05.665366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cfce0-e39d-46e1-8d97-e7bc9ec5026b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:05.699071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:05.699691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:05.699975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:05.700242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:05.711013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:05.715873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:05.717504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:05.718661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 599/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:29:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:05.723677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:05.821515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dc2a3-825c-468f-b1e7-2d8de8629c71 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:05.845123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:05.855286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:05.855911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:05.856176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:05.856448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:05.857137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:05.857438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:05.858889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:05.866467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 624/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:29:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:05.872280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:05.919039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:05.977043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99d92f-e3ad-45aa-b39b-c0762bb1b3da None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.009527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.010165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.010438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.010704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.020966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.022416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:06.036991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:06.038131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:06.039145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:06.039145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:06.039145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:06.039145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 600/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:06.044146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.144206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65c1f4-677e-4b91-a884-2f5eba351918 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.176606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.177203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.177489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.177759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.178460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:06.178878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.180230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.193554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 625/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:06.199330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.303347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:06.309713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897f2cf-3b75-4986-addb-62a61eb369f5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.342624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.343238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.343511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.343783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.350875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.352315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:06.363122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:06.364163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:06.365496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 601/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:06.370235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.416860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:06.468703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbb7ef-57a2-408a-9d71-b27a047b4412 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.502046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.502669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.502948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.503295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.503976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:06.504287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.505693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.522209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 626/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:06.528061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.624973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccef2af-fdad-41a5-b2ee-5ebd121db2ce None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.656821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.657431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.657713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.657985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.668853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.670795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:06.684347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:06.686000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:06.687095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:06.687095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:06.687095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:06.687095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 602/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:06.691806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.786753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd6ce8-4a0d-4d6c-b419-73fb90e7045f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.788971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:06.819547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.820140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.820417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.820687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.821322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:06.821622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.822991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.833698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 627/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:06.839758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:06.849380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:06.936278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16d42-8095-424c-b795-91ad5585aa74 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:06.969476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:06.970123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:06.970409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:06.970678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:06.977252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:06.978743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:06.990493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:06.991535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:06.992475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 603/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:06.997584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:07.096909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97dfb9-c562-4147-846d-25c46ba8a458 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:07.130096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:07.130695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:07.130973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:07.131243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:07.131928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:07.132231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:07.133633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0605665-a7de-447c-a017-69f6cc440c66 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:07.145467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 628/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:07.151425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:07.216733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:07.251370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6334-e81d-4e30-88a2-76da32c0910d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:07.283848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:07.284486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:07.284757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:07.285046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/fa0c95e5281f4f9d9dac910415d10932/roles/e5423f9e4352439ba0663085cd54ced6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:07.298593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=fa0c95e5281f4f9d9dac910415d10932, role_id=e5423f9e4352439ba0663085cd54ced6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:07.300017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab79c7-aa53-4c0d-9b96-1f980c897621 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:07.458479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 604/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:29:07 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/fa0c95e5281f4f9d9dac910415d10932/roles/e5423f9e4352439ba0663085cd54ced6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:29:09.587113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21467c-e8cf-4ccf-8bc2-3230170f5638 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:09.587662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21467c-e8cf-4ccf-8bc2-3230170f5638 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:09.587662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21467c-e8cf-4ccf-8bc2-3230170f5638 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:09.589077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 629/1233] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:29:09.594223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:09.594627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:09.594772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:09.612873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:09.614087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:09.678532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 605/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:09.949607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21467c-e8cf-4ccf-8bc2-3230170f5638 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:09.974995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:10.049032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21467c-e8cf-4ccf-8bc2-3230170f5638 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.081794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.082447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.082725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.083016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.093680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:10.098587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:10.099761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:10.100799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 630/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Sep 9 11:29:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:10.105736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.151337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:10.202934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a4ff2-f934-4b0d-8526-0b5d1947f4e7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.236511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.237122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.237396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.237666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.238359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:10.238661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:10.240019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:10.248192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 606/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:10.254242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.338065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:10.353809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79582d7e-b0fe-4c4d-9d12-180a84cbd9b3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.386750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.387375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.387652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.388442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.398701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:10.400230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:10.414996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:10.416133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:10.417075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:10.417075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:10.417075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:10.417075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 631/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:10.422081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.519966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96115c-9a33-4c32-be11-eec08ffbe65f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.528468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:10.552774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.553369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.553647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.553919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.554587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:10.554891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:10.556253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:10.567784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 607/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:10.573478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.672340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323700a-aebd-42e2-a67f-a25b0209b58a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.705795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.706424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.706707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.706989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.709654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:10.714707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:10.716196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:10.727171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:10.728284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:10.729433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 632/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:10.734380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.830730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d903-6a6d-4014-b38f-879cf2591c8e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:10.863604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:10.864191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:10.864468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:10.864737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:10.865467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:10.865765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:10.867216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:10.884186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 608/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:10.891637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:10.990488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee396d82-a644-4c2d-a3d8-70d64d3dc783 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.023201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.023798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.024074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.024345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.083733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:11.090274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.091700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.185283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 633/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:29:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:29:11.191102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:11.218581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:11.287656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365cfd5-212b-41f0-bf12-5938610d1296 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.321329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.321952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.322255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.322527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.333140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.334658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:11.349917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:11.351118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:11.352408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:11.352408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:11.352408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:11.352408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 609/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:11.356937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:11.452370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:11.458014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c3a52-4ac7-400d-abde-9ccc7c95e025 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.490193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.490787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.491065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.491338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.492067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:11.492370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.493812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.505227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 634/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:11.511010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:11.588777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:11.609160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149871d-6e8e-484a-b44f-6f25b74ebc4a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.642026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.642659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.642937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.643209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.649618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.651076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:11.662412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:11.663318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:11.664565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 610/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:11.669459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:11.767530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16affa1e-3ca6-4706-beb3-438ae1b7b46a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.800993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.801608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.801890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.802194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.802885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:11.803188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.804595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982a7f6-4d7b-4a6f-9eef-33bc81b358b3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.816882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 635/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:11.823253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:11.922212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3a8f-3685-4eb6-a753-7390fd4662a7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:11.958366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:11.958959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:11.959235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:11.959505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:11.960514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] RBAC: Adding query filter params (role.id=e5423f9e4352439ba0663085cd54ced6, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=fa0c95e5281f4f9d9dac910415d10932) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:11.960818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:11.962259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad442ff9-f726-47b3-a828-fbac5765c630 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:11.967908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 611/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:29:11 2022] GET /identity/v3/role_assignments?user.id=fa0c95e5281f4f9d9dac910415d10932&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=e5423f9e4352439ba0663085cd54ced6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:14.152102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a7c8-8138-4df1-8935-41a3815f51cb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:14.152658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a7c8-8138-4df1-8935-41a3815f51cb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:14.152658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a7c8-8138-4df1-8935-41a3815f51cb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:14.154585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 636/1247] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:14.159679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:14.160072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:14.160228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:14.166645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:14.178640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:14.179827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:14.243956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 612/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:14.517471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4a7c8-8138-4df1-8935-41a3815f51cb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:14.535124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:14.614914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4a7c8-8138-4df1-8935-41a3815f51cb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:14.647443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:14.648052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:14.648323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:14.648589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:14.658259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:14.664410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:14.665474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:14.666558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 637/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:29:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:14.670001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:14.769443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d06357-decd-4d27-ab82-24d71e23b70d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:14.797930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:14.803497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:14.804097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:14.804376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:14.804646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:14.805299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:14.805597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:14.807033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:14.815406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 613/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:29:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:14.821317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:14.897991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:14.921136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e9258-83da-4686-ae39-4d89263488d5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:14.953242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:14.953849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:14.954147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:14.954416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:14.964687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:14.966149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:14.979834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:14.985217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:14.986390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:14.986390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:14.986390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:14.986390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 638/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:14.988669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:15.085886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9111cf1-50a8-448b-b390-fc439333cc61 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:15.119800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:15.120479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:15.120775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:15.121045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:15.121672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:15.121993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:15.123373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:15.134947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 614/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:15.140706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:15.238108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23adc75-3b38-406c-926d-eae720511bd4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:15.262584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:15.269980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:15.270627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:15.270903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:15.271176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:15.278904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:15.280317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:15.291067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:15.292256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:15.293551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 639/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:29:15.298551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:15.353477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:15.440360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689045-ac19-4123-a2b4-5866f8f524c7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:15.440360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:15.440360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:15.440360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:15.440811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:15.440811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:15.440811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:15.440811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:15.454435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 615/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:15.460635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:15.558596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82346a-d1c5-446d-8ebc-4cce475c288a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:15.592103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:15.592697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:15.592970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:15.593255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:15.645633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:15.657825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:15.659302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:15.752776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 640/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:29:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:29:15.758781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:15.857386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421938a7-ed48-45e1-84dd-433978e2483d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:15.870693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:15.891420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:15.892040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:15.892313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:15.892594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:15.902275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:15.903733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:15.917795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:15.919083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:15.920430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:15.920430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:15.920430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:15.920430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 616/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:15.924748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:16.011028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:16.023116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7d5b2-d46c-47fa-8f9d-6ab3e7a6af60 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:16.055728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:16.056337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:16.056629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:16.056912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:16.057560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:16.057863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:16.059249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:16.070574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 641/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:16.077016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:16.176824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461bb2-fd53-431c-9a67-8f62ab30cbba None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:16.209899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:16.210792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:16.211073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:16.211346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:16.217787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:16.219278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:16.230291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:16.231897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:16.232866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 617/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:16.237885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:16.337308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3311-6d4b-4188-be1e-dac463237c5c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:16.370880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:16.371489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:16.371765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:16.372039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:16.372727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:16.373035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:16.374537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2427fe-7d24-4b00-9149-7646edf7c10e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:16.386385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 642/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:16.393141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:16.419251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:16.491347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c320-b2ef-401c-b86a-24a7665163f1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:16.523724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:16.524319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:16.524600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:16.524871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:16.525587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=fa0c95e5281f4f9d9dac910415d10932) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:16.525894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:16.527353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ae8a-01ce-42e1-90d5-34dc95333776 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:16.533246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 618/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:29:16 2022] GET /identity/v3/role_assignments?user.id=fa0c95e5281f4f9d9dac910415d10932&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:18.667193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fde706-0951-4a7d-9cf6-eee56e4b52e9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:18.667708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fde706-0951-4a7d-9cf6-eee56e4b52e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:18.667708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fde706-0951-4a7d-9cf6-eee56e4b52e9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:18.669125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 643/1261] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:18.674385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:18.674801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:18.674932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:18.692621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:18.693817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:18.760483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 619/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:19.033794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fde706-0951-4a7d-9cf6-eee56e4b52e9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.045143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.133751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fde706-0951-4a7d-9cf6-eee56e4b52e9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.167694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.168353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.168633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.168905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.178787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:19.183555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:19.191476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:19.192453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 644/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:19.193181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.210183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.290884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399e1a-6a6e-4346-8d0e-ce75cbd2db1a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.324142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.324775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.325055 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.325327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.325990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:19.330891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:19.330891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:19.335669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 620/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:19.342893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.412796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.445718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a00147-2c2d-4a4b-9b33-545469172aaa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.478418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.479170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.479542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.479911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.490362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:19.491787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:19.507222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:19.508419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:19.509713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:19.509713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:19.509713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:19.509713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 645/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:19.514328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.595612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.614273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad30bb-e5f3-4591-a449-16f49c1160c8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.645611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.646287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.646557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.646820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.647440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:19.647740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:19.649136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:19.661038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 621/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:19.666769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.764814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b086513-f79b-4453-becd-83b9b0fd86b7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.784369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.799549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.800173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.800453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.800726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.808964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:19.810424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:19.821150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:19.822339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:19.823745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 646/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:19.828169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:19.926830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61c1a7-74dc-42f9-98bf-f7f83c9958a0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:19.959323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:19.960187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:19.960490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:19.960757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:19.961472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:19.961792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:19.963262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:19.973069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:19.981157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 622/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:19.986993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:20.084327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b969c7e-4e6a-4fc4-b42d-79256c9b76c5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:20.118848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:20.119470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:20.119747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:20.120019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:20.130422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:20.131836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:20.145990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:20.147147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:20.148394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:20.148394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:20.148394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:20.148394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 647/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:20.152831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:20.251613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b2fbb-cc34-4053-b358-0d4ac3b72e9f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:20.286626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:20.287226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:20.287505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:20.287777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:20.288461 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:20.288763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:20.290197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:20.306366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 623/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:20.312477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:20.346145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:20.413712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3829a-5616-4470-807d-93747033c2eb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:20.446098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:20.446734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:20.447239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:20.447699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:20.455558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:20.456991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:20.468250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:20.469405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:20.470462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 648/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:20.475294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:20.533659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:20.574454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69f324-0e57-4555-beec-f23d1cfc1a91 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:20.607714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:20.608305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:20.608578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:20.608844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:20.609579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:20.609884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:20.611324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8aacc-8c0f-4638-ab43-766bc464716e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:20.623774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 624/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:20.633767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:20.724029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:20.732853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4003b9-b70f-4d20-8c47-008ec2a20729 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:20.766228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:20.766870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:20.767146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:20.767529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] PATH_INFO: `/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/fa0c95e5281f4f9d9dac910415d10932/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:20.769396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] RBAC: Authorizing `identity:create_grant(project_id=e972aecf7fdf4696814c61481ddcca9e, user_id=fa0c95e5281f4f9d9dac910415d10932, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:20.770836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb38104-08b2-41f9-9cbf-0117d95325ab None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:20.904672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 649/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Sep 9 11:29:20 2022] PUT /identity/v3/projects/e972aecf7fdf4696814c61481ddcca9e/users/fa0c95e5281f4f9d9dac910415d10932/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:29:23.082277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df922004-5938-4f33-b365-338f0139fde2 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:23.082837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df922004-5938-4f33-b365-338f0139fde2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:23.082837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df922004-5938-4f33-b365-338f0139fde2 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:23.084146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 625/1274] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:23.089645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:23.090146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:23.090264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:23.109000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:23.110227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:23.175982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 650/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:23.444532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df922004-5938-4f33-b365-338f0139fde2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:23.546728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df922004-5938-4f33-b365-338f0139fde2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:23.564619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:23.581361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:23.581984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:23.582293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:23.582565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:23.592588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:29:23.597391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:23.598532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:23.599638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 626/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Sep 9 11:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:23.604580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:23.683698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:23.701862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe70cc-9d82-49c6-b3dd-fdf69cd85b62 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:23.734321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:23.734919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:23.735181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:23.735449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:23.736098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:23.736394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:23.737751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:23.745421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 651/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Sep 9 11:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:23.751821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:23.848928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75300e-923e-49df-82f1-6d495df5236f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:23.883989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:23.884607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:23.884886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:23.885156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:23.894771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:23.896194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:23.910184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:23.916072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:23.918984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:23.918984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:23.918984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:23.918984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 627/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:23.919224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.019248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943264-f302-450c-a499-9bb3f6966482 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:24.054772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:24.055367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:24.055642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:24.055910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:24.056538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:24.056834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:24.058333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:24.070473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 652/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:24.076386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.107962 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:24.180300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5914f2b-2152-4112-a1ed-7e13da29d865 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:24.212459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:24.217096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:24.217096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:24.217096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:24.222473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:24.223953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:29:24.235623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:24.236728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:24.237680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 628/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:24.242644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.246443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:24.338736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f64f-6cee-4c67-90ad-f3d4d8116f94 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:24.371997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:24.372601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:24.372882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:24.373153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:24.373880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:24.374221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:24.375640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:24.400812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 653/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Sep 9 11:29:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:24.407353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.495562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:24.504905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b92c6-c420-4e9e-990c-480d105ff3b1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:24.538099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:24.538702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:24.538982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:24.539256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:24.604098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:24.605475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:24.691303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:24.701762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 629/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Sep 9 11:29:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:29:24.707955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.768622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:24.804992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc970e5c-7314-495a-b2ec-e72527e5204b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:24.837776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:24.838420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:24.838701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:24.838972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:24.849034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:24.850503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:29:24.865106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:24.871103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:29:24.872535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:29:24.872535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:29:24.872535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:24.872535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 654/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:24.872764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:24.970415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50981e6-3c32-4c34-9b4d-7615735b20e7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:25.003430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:25.004023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:25.004301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:25.004572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:25.005210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:25.005522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:25.006901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:25.018832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 630/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Sep 9 11:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:25.024720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:25.122924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e65a6-474c-4184-8202-9e97927bc347 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:25.135939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:25.155918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:25.156559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:25.156836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:25.157107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:25.163577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:25.164991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:29:25.175804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:25.176862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 09 11:29:25.177973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 655/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Sep 9 11:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:25.183027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:25.232763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:25.280694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ee634-0f30-4546-92d8-8a1f15c0af1d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:25.314585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:25.315197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:25.315465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:25.315737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:25.316418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:25.316719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:25.318162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de9ec2-8226-46b8-b0b9-1c35a642b094 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:25.329784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 631/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Sep 9 11:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:25.336486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:25.435734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef5c9-0010-47ca-9470-8ed9083e0112 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:25.467881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:25.468478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:25.468758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:25.469023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:25.469731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] RBAC: Adding query filter params (role.id=99f6f437de8e4cf2810a3bd858413db3, scope.project.id=e972aecf7fdf4696814c61481ddcca9e, user.id=fa0c95e5281f4f9d9dac910415d10932) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:25.470032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:25.471456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978e9a8-f56e-4ce9-8c6f-87c418875c00 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:25.477114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 656/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Sep 9 11:29:25 2022] GET /identity/v3/role_assignments?user.id=fa0c95e5281f4f9d9dac910415d10932&scope.project.id=e972aecf7fdf4696814c61481ddcca9e&role.id=99f6f437de8e4cf2810a3bd858413db3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:27.642864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08c80-616d-4eea-ac21-d40e95d86d06 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:27.643373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08c80-616d-4eea-ac21-d40e95d86d06 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:27.643373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f08c80-616d-4eea-ac21-d40e95d86d06 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:27.644651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 632/1288] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:27.650143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:27.650554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:27.650697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:27.669005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:27.670225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:27.698973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:27.734947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 657/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:28.010841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f08c80-616d-4eea-ac21-d40e95d86d06 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:28.112888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f08c80-616d-4eea-ac21-d40e95d86d06 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:28.124213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:28.147144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:28.147794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:28.148098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:28.148368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:28.148945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:28.150333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:28.158948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:28.160164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:28.161046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 633/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:29:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:28.166133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:28.262592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67638fdd-ee65-49c8-bf2b-93c16e5c3cea None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:28.295299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:28.295898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:28.296179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:28.296452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:28.297083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:28.297386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:28.298771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804c4df-62f5-4a62-bb11-015c1e2c67c7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:28.303404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 658/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Sep 9 11:29:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:28.309966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:28.407860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a027e-7cf8-4829-a047-ab9340d16a6c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:28.441043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:28.441639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:28.441918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:28.442221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:28.442844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:28.443150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:28.444508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4b0d1-8800-4a1b-b091-4cc5b1043991 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:28.448835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 634/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Sep 9 11:29:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:30.595124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bd18a-b067-44e6-8e82-046a12497a94 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:30.595695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bd18a-b067-44e6-8e82-046a12497a94 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:30.595695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bd18a-b067-44e6-8e82-046a12497a94 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:30.596981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 659/1293] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:30.602739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1686-af1a-48ca-bac3-28e82d1d504e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:30.603164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1686-af1a-48ca-bac3-28e82d1d504e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:30.603276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d1686-af1a-48ca-bac3-28e82d1d504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:30.623008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-974d1686-af1a-48ca-bac3-28e82d1d504e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:30.624181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-974d1686-af1a-48ca-bac3-28e82d1d504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:30.653344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:30.688167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 635/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:29:30.962848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bd18a-b067-44e6-8e82-046a12497a94 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:30.994777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:31.065392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bd18a-b067-44e6-8e82-046a12497a94 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:31.098262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:31.098855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:31.099129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:31.099395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:31.100040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:31.101382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1611d-38f4-43d3-8c28-7e7b7e4ea624 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:31.195776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 660/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Sep 9 11:29:30 2022] POST /identity/v3/services => generated 246 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:29:33.364367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07dab0-ff27-4154-81ef-e644627b07bf None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:33.364919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07dab0-ff27-4154-81ef-e644627b07bf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:33.364919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07dab0-ff27-4154-81ef-e644627b07bf None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:33.366291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 636/1296] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:33.371540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:33.371944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:33.372091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:33.389914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:33.391135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:33.461702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 661/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:29:33.732177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07dab0-ff27-4154-81ef-e644627b07bf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:33.833155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07dab0-ff27-4154-81ef-e644627b07bf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:33.867709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:33.868796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:33.868796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:33.869024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:33.869482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:33.870887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:33.879521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:33.883115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:33.884212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 637/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:29:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:33.886849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:33.905159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:33.983800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc81a6d-1e15-4ad5-92f2-1a30b404b7c6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:34.017162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:34.017758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:34.018038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:34.018339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:34.018966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:34.019265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:34.020656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:34.025816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 662/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Sep 9 11:29:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:29:34.032534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:34.066787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:34.135073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31551c2-a4e0-4b50-b4b6-68828fb9473a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:34.168721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:34.169323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:34.169598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:34.169866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:34.170585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] RBAC: Adding query filter params (interface=public, service_id=62527059443140fbba6f30d3912e50aa, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:34.171101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:34.172498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d3efc-1b8f-46fc-beb9-ec852261b1b7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:34.177477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 638/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Sep 9 11:29:34 2022] GET /identity/v3/endpoints?service_id=62527059443140fbba6f30d3912e50aa&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:34.183180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:34.268345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:34.286739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7593e64-09a0-4927-b23c-23471575412a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:34.319907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:34.320516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:34.320797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:34.321069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:34.321675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:34.323075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ef7b-a5da-4b92-a409-9dcc94837db2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:34.330116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 663/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Sep 9 11:29:34 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:29:36.509865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f1e6d-b43f-4732-9f57-d7e342cd2057 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:36.510415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f1e6d-b43f-4732-9f57-d7e342cd2057 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:36.510415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f1e6d-b43f-4732-9f57-d7e342cd2057 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:36.511834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 639/1302] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:29:36.517147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:36.517581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:36.517676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:36.535443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:29:36.536629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:29:36.601219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 664/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:29:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:29:36.872089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f1e6d-b43f-4732-9f57-d7e342cd2057 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:36.966485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:36.973324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f1e6d-b43f-4732-9f57-d7e342cd2057 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:37.006478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:37.007100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:37.007379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:37.007651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:37.008231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:37.009612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:29:37.018958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:29:37.023354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 09 11:29:37.024409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 640/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Sep 9 11:29:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:29:37.026662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:37.101162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:37.124316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1cd7b-2675-4a24-95bb-8b0140d51536 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:37.157169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:37.157744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:37.158024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:37.158330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:37.158965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:29:37.159267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:37.160622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee70326-a6d3-401b-a428-41ee82070830 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:37.165500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 665/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Sep 9 11:29:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:29:37.171385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:29:37.269800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b27f-854c-4ad7-8957-9a0af3561126 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:29:37.302306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:29:37.302899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:29:37.303173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:29:37.303442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:29:37.304095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:29:37.305454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ab526-4af6-45bb-a576-febd11f24f1c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:29:37.401067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 641/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:29:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:29:47.201034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=30 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:29:47.396345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:03.052123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aab-422e-4a68-972d-297aa9936fe7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:03.053027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aab-422e-4a68-972d-297aa9936fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:03.053463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8340aab-422e-4a68-972d-297aa9936fe7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:03.061813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2baec-400d-4cd8-8df0-d7a0afcdfabf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:03.062784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2baec-400d-4cd8-8df0-d7a0afcdfabf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:03.063206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2baec-400d-4cd8-8df0-d7a0afcdfabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:03.068333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 666/1307] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:03 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:03.204862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-76d2baec-400d-4cd8-8df0-d7a0afcdfabf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:03.209335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-76d2baec-400d-4cd8-8df0-d7a0afcdfabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:03.438757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 642/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:37:03.728800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:03.729686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:03.729928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:03.731725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 667/1309] 10.222.0.60 () {54 vars in 965 bytes} [Fri Sep 9 11:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:03.738697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929b27b-8fab-49b0-b2e9-78eba043aa7f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:03.739055 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929b27b-8fab-49b0-b2e9-78eba043aa7f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:03.739260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929b27b-8fab-49b0-b2e9-78eba043aa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:03.830392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8929b27b-8fab-49b0-b2e9-78eba043aa7f None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:03.832254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8929b27b-8fab-49b0-b2e9-78eba043aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:03.856856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:03.928626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 643/1310] 10.222.0.60 () {58 vars in 1062 bytes} [Fri Sep 9 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:03.937937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:03.940107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:04.067800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6a069-6d8b-49a3-93ec-bd521e757801 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:04.157123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:04.158137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:04.158650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:04.159142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:04.161065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:04.221224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:04.289653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:04.291775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908656c-0fd8-4f3c-8227-5d0e64a77b56 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:04.315837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 668/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:06.875422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:06.875927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:06.875927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:06.877217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 644/1312] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:06.882471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d770ed-0441-4296-a620-4e26e3b34ded None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:06.882850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d770ed-0441-4296-a620-4e26e3b34ded None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:06.883024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d770ed-0441-4296-a620-4e26e3b34ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:06.962556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-81d770ed-0441-4296-a620-4e26e3b34ded None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:06.963738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-81d770ed-0441-4296-a620-4e26e3b34ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:07.056091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 669/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:07.338612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:07.340162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:07.443345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ac27d-8aa1-4521-841a-ede825ff0759 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:07.513680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:07.515920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:07.516519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:07.516795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:07.517063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:07.518705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:07.629181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:07.630657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ba936-e129-4226-9969-57ec6d756c03 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:07.649849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 645/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:10.276080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:10.276642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:10.276642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:10.277943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 670/1315] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:10.283100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef58d-edea-41d7-8212-5fd68d4bd572 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:10.283540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef58d-edea-41d7-8212-5fd68d4bd572 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:10.283660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef58d-edea-41d7-8212-5fd68d4bd572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:10.347026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f13ef58d-edea-41d7-8212-5fd68d4bd572 None None] MFA Rules not processed for user `59019ff482684682b14c5cdbb421e5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:10.348213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f13ef58d-edea-41d7-8212-5fd68d4bd572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:10.398890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:10.442600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 646/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:10.731400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:10.732985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:10.742671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:10.835205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637408cb-8c2f-4cdd-afd3-cb0dd909a68d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:10.908931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:10.909517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:10.909798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:10.910094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:10.911666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:10.972288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:11.017170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:11.018623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7fe2c-2b0b-4159-af2e-3c5de5f5f19c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:11.037228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 671/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:13.983578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9643a6-bd69-435a-87ab-416729b819aa None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:13.984119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9643a6-bd69-435a-87ab-416729b819aa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:13.984119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9643a6-bd69-435a-87ab-416729b819aa None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:13.985439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 647/1318] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:13.990788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d91b1-e75b-4c79-8e33-56c0a9dd43d7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:13.991198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d91b1-e75b-4c79-8e33-56c0a9dd43d7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:13.991323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d91b1-e75b-4c79-8e33-56c0a9dd43d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:14.055943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7f6d91b1-e75b-4c79-8e33-56c0a9dd43d7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:14.057665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7f6d91b1-e75b-4c79-8e33-56c0a9dd43d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:14.151577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 672/1319] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:14.449027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:14.449284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:14.449550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:14.450920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 648/1320] 10.222.0.60 () {54 vars in 965 bytes} [Fri Sep 9 11:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:14.456571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22e22-46f9-4df8-bab6-fca985c53388 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:14.456852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22e22-46f9-4df8-bab6-fca985c53388 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:14.457119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22e22-46f9-4df8-bab6-fca985c53388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:14.479287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:14.531893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-74a22e22-46f9-4df8-bab6-fca985c53388 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:14.533070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-74a22e22-46f9-4df8-bab6-fca985c53388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:14.609144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 673/1321] 10.222.0.60 () {58 vars in 1062 bytes} [Fri Sep 9 11:37:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:14.618297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:14.619933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:14.720874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d8876-4bf0-4bca-82f2-ec25c5b7507d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:14.793541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:14.794230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:14.794438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:14.794706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:14.796298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:14.803088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:14.904163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:14.905586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569afd83-c603-463d-b326-7c8cff155be3 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:14.923684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 649/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:18.433177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:18.433698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:18.433816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:18.435234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 674/1323] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:18.440531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8e0b-2e59-4bb8-b756-d39423e3db5d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:18.440984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8e0b-2e59-4bb8-b756-d39423e3db5d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:18.441080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8e0b-2e59-4bb8-b756-d39423e3db5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:18.508586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-55fe8e0b-2e59-4bb8-b756-d39423e3db5d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:18.509834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-55fe8e0b-2e59-4bb8-b756-d39423e3db5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:18.572332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:18.616973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 650/1324] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:18.903208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:18.904740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:19.008553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defe98c-facd-472e-9a36-729af6046d9f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:19.021116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:19.086694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:19.087305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:19.087613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:19.088481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:19.090169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:19.203313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:19.204748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b138-d327-414e-a5a5-9b7f2fb969ad service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:19.222465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 675/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:21.441441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:21.442019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:21.442317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:21.443761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 651/1326] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:21.450594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcacae-b078-465c-ad1c-2e77b8696c38 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:21.451152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcacae-b078-465c-ad1c-2e77b8696c38 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:21.451152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcacae-b078-465c-ad1c-2e77b8696c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:21.502406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:21.520603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-97fcacae-b078-465c-ad1c-2e77b8696c38 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:21.523525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-97fcacae-b078-465c-ad1c-2e77b8696c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:21.619736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 676/1327] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:21.909947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:21.911544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:22.014391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe57fb7-28d2-413e-9b65-0820c4619d99 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:22.088856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:22.089458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:22.089735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:22.090005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:22.091630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:22.108358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:22.201453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:22.202841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee2e0-29ce-4d1b-a2bc-409882ca6f50 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:22.221191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 652/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:24.444973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:24.445476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:24.445592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:24.446935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 677/1329] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:24.452639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726d18-e0f5-4e3f-9e15-3d359dbb0d8e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:24.453039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726d18-e0f5-4e3f-9e15-3d359dbb0d8e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:24.453188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726d18-e0f5-4e3f-9e15-3d359dbb0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:24.519390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8e726d18-e0f5-4e3f-9e15-3d359dbb0d8e None None] MFA Rules not processed for user `59019ff482684682b14c5cdbb421e5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:24.520544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8e726d18-e0f5-4e3f-9e15-3d359dbb0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:24.585431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:24.616537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 653/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:24.904675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:24.906263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:25.031039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b172b2-1227-4964-bb87-81ed4a924a35 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:25.056339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:25.116781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:25.117312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:25.117591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:25.117862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:25.120032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:25.228109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:25.229830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96250d1c-0d32-4a27-b97c-ca25c680589a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:25.247995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 678/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:27.944768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:27.945304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:27.945304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:27.946706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 654/1332] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:27.952171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126ad60-5bd3-4e79-a99b-df0404a4e8eb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:27.952630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126ad60-5bd3-4e79-a99b-df0404a4e8eb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:27.952731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126ad60-5bd3-4e79-a99b-df0404a4e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:28.016722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:28.019596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c126ad60-5bd3-4e79-a99b-df0404a4e8eb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:28.020818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c126ad60-5bd3-4e79-a99b-df0404a4e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:28.131746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 679/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:28.412517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:28.414085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:28.515895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038b76-0170-41be-8f90-10cceec552d7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:28.587853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:28.588459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:28.588736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:28.589004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:28.590641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:28.617112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:28.697650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:28.699353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9f46-f2ed-401e-b537-76b06e705786 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:28.717817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 655/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:30.927470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:30.928001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:30.928001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:30.929329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 680/1335] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:30.934632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024c964-3024-4562-8484-a550a3259c55 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:30.935076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024c964-3024-4562-8484-a550a3259c55 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:30.935180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024c964-3024-4562-8484-a550a3259c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:31.000625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8024c964-3024-4562-8484-a550a3259c55 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:31.001826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8024c964-3024-4562-8484-a550a3259c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:31.074494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:31.096354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 656/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:31.376334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:31.383073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:31.478718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ab83-7467-4639-96db-ee511787a793 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:31.506815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:31.551097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:31.551688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:31.551969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:31.552242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:31.553822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:31.660387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:31.661735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085529c-7cc4-45fb-86fd-39e40f6a8418 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:31.681136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 681/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:34.487008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:34.487562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:34.487562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:34.488908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 657/1338] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:34.494191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23f4ae-a6c2-477f-b6fa-1b8ae9aeab1d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:34.494641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23f4ae-a6c2-477f-b6fa-1b8ae9aeab1d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:34.494758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23f4ae-a6c2-477f-b6fa-1b8ae9aeab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:34.559841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7e23f4ae-a6c2-477f-b6fa-1b8ae9aeab1d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:34.561043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7e23f4ae-a6c2-477f-b6fa-1b8ae9aeab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:34.569611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:34.659327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 682/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:37:34.942388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:34.943952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:35.047140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2889d7-6880-4f28-bead-18fd4cfb0fc5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:35.119860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:35.120456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:35.120735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:35.121003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:35.122615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:35.157364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:35.229032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:35.230408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3188f-ada3-48a7-96d4-7f081e717b18 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:35.248272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 658/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:39.638411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:39.638945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:39.639108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:39.640496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 683/1341] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:39.645791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee17a2-6a89-4ce6-b3c8-add3fa102902 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:39.646274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee17a2-6a89-4ce6-b3c8-add3fa102902 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:39.646373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee17a2-6a89-4ce6-b3c8-add3fa102902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:39.712402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d9ee17a2-6a89-4ce6-b3c8-add3fa102902 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:39.713595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d9ee17a2-6a89-4ce6-b3c8-add3fa102902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:39.806957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:39.824320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 659/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:40.112732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:40.114299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:40.215144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a773e-0862-4058-a581-3e45b2cf4e62 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:40.252599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:40.288817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:40.289422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:40.289716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:40.290585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:40.292175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:40.403704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:40.405356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66d072-e15a-45a1-9611-5c21c9056339 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:40.424635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 684/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:43.598797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:43.599329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:43.599446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:43.600791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 660/1344] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:43.606619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10ad6-b149-452e-a90d-e81c4d4ad931 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:43.607043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10ad6-b149-452e-a90d-e81c4d4ad931 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:43.607281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10ad6-b149-452e-a90d-e81c4d4ad931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:43.678307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2de10ad6-b149-452e-a90d-e81c4d4ad931 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:43.679418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2de10ad6-b149-452e-a90d-e81c4d4ad931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:43.698755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:43.777272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 685/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:44.062631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:44.064162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:44.168881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daef59-bf57-4fb4-9839-90031008a63f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:44.241527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:44.242163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:44.242446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:44.242740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:44.244345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:44.286624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:44.373345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:44.374872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ff702-1ae2-4170-a69f-dbec2c1a4dc7 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:44.395345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 661/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:48.947162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:48.947740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:48.947740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:48.949376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 686/1347] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:48.955203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6c52-9a58-4bcd-96a7-eb879d7e5337 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:48.955677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6c52-9a58-4bcd-96a7-eb879d7e5337 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:48.955965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6c52-9a58-4bcd-96a7-eb879d7e5337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:49.028183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bdaa6c52-9a58-4bcd-96a7-eb879d7e5337 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:49.029426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bdaa6c52-9a58-4bcd-96a7-eb879d7e5337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:49.132361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:49.134996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 662/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:49.417557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:37:49.419145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:49.521280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0853511-5995-4d7e-8b11-4931a4bf1196 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:37:49.572526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:49.597789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:37:49.598421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:49.598725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:49.599018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:49.600608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:49.707854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:37:49.709192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a8ecd-b896-4ece-bc27-0137016466d0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:37:49.727530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 687/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:37:54.674514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc7d0-b082-4b06-afbc-c17fbd5ea8cf None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:54.674514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc7d0-b082-4b06-afbc-c17fbd5ea8cf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:54.674514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc7d0-b082-4b06-afbc-c17fbd5ea8cf None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:54.674514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 663/1350] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:54.679289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14214-96a8-4ddf-9ae2-ddd1bca189e6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:54.679800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14214-96a8-4ddf-9ae2-ddd1bca189e6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:54.679908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d14214-96a8-4ddf-9ae2-ddd1bca189e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:54.770404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-04d14214-96a8-4ddf-9ae2-ddd1bca189e6 None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:54.771715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-04d14214-96a8-4ddf-9ae2-ddd1bca189e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:54.792342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:54.936812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 688/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:37:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:37:59.034280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026d98-10d6-4d36-93f0-de64957d0b52 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:59.035387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026d98-10d6-4d36-93f0-de64957d0b52 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:59.035801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026d98-10d6-4d36-93f0-de64957d0b52 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:59.037307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 664/1352] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:37:59.043371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f527d-5452-4dba-ad49-31bff348094b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:37:59.043825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f527d-5452-4dba-ad49-31bff348094b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:37:59.043916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f527d-5452-4dba-ad49-31bff348094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:37:59.128588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8a5f527d-5452-4dba-ad49-31bff348094b None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:37:59.130495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8a5f527d-5452-4dba-ad49-31bff348094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:37:59.212732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:37:59.252132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 689/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:37:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:00.708789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ea2d-a800-4ebf-850a-e7c9a3be6227 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:00.709326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ea2d-a800-4ebf-850a-e7c9a3be6227 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:00.709326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1ea2d-a800-4ebf-850a-e7c9a3be6227 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:00.711231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 665/1354] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:00.716505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ce4c-73f3-471a-93fa-e87a74879c48 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:00.716928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ce4c-73f3-471a-93fa-e87a74879c48 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:00.717068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ce4c-73f3-471a-93fa-e87a74879c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:00.791969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2b83ce4c-73f3-471a-93fa-e87a74879c48 None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:00.793154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2b83ce4c-73f3-471a-93fa-e87a74879c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:00.901762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 690/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:38:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:04.452614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c1c0-0ba2-472e-96e8-d83740685b0c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:04.453179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c1c0-0ba2-472e-96e8-d83740685b0c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:04.453179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c1c0-0ba2-472e-96e8-d83740685b0c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:04.454509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 666/1356] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:04.460068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761bb7-c796-41cc-ba92-7ea8cf8dff44 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:04.460961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761bb7-c796-41cc-ba92-7ea8cf8dff44 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:04.461242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3761bb7-c796-41cc-ba92-7ea8cf8dff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:04.500684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:04.531756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d3761bb7-c796-41cc-ba92-7ea8cf8dff44 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:04.532936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d3761bb7-c796-41cc-ba92-7ea8cf8dff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:04.634276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 691/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:04.959660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:04.960058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:04.960933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:04.963217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 667/1358] 10.222.0.60 () {54 vars in 962 bytes} [Fri Sep 9 11:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:04.969119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3deab1-cabd-46db-95cc-f4e4b438ff67 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:04.969399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3deab1-cabd-46db-95cc-f4e4b438ff67 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:04.969662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3deab1-cabd-46db-95cc-f4e4b438ff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:05.057683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1b3deab1-cabd-46db-95cc-f4e4b438ff67 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:05.058928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1b3deab1-cabd-46db-95cc-f4e4b438ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:05.114329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:05.186099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 692/1359] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Sep 9 11:38:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:05.193963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:05.195594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:05.349831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d153-8247-4998-82f6-3686751d20e3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:05.436417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:05.481318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:05.482085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:05.482475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:05.482833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:05.484531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:05.583061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:05.584827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2e46-9073-4ab0-8cf7-4c86bd966924 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:05.604910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 668/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:07.972459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:07.972994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:07.972994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:07.974294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 693/1361] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:07.980256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80252f1e-0ecc-4e3e-aff7-b87d22b17991 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:07.980675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80252f1e-0ecc-4e3e-aff7-b87d22b17991 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:07.980802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80252f1e-0ecc-4e3e-aff7-b87d22b17991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:08.076953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-80252f1e-0ecc-4e3e-aff7-b87d22b17991 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:08.078227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-80252f1e-0ecc-4e3e-aff7-b87d22b17991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:08.117905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:08.180854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 669/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:08.471392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:08.472933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:08.646317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8b9e-393f-4ae2-b4da-97d080c528e8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:08.660241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:08.791697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:08.792346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:08.792618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:08.792884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:08.794528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:08.890632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:08.892228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa52eb-f09e-44d0-bc35-7449048b0edd service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:08.911392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 694/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:11.563375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539eb3-2a0c-4856-bad8-d8a68bca98ae None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:11.563928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539eb3-2a0c-4856-bad8-d8a68bca98ae None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:11.563928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539eb3-2a0c-4856-bad8-d8a68bca98ae None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:11.565306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 670/1364] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:11.570612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae21ad-18a1-4cce-baf0-3f320470d005 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:11.571001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae21ad-18a1-4cce-baf0-3f320470d005 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:11.571178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae21ad-18a1-4cce-baf0-3f320470d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:11.587502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:11.642539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dbae21ad-18a1-4cce-baf0-3f320470d005 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:11.643663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dbae21ad-18a1-4cce-baf0-3f320470d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:11.747575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 695/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:12.041261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:12.041546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:12.042006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:12.043380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 671/1366] 10.222.0.60 () {54 vars in 962 bytes} [Fri Sep 9 11:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:12.049032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d4d-5680-4231-86e0-66ce12196396 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:12.049724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d4d-5680-4231-86e0-66ce12196396 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:12.050253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d4d-5680-4231-86e0-66ce12196396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:12.132422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-53af3d4d-5680-4231-86e0-66ce12196396 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:12.134079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-53af3d4d-5680-4231-86e0-66ce12196396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:12.170266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:12.256970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 696/1367] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Sep 9 11:38:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:12.264931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:12.266580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:12.434970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71774-53dd-4806-8468-e8da83f40d12 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:12.457039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:12.561105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:12.561719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:12.561994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:12.562293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:12.563951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:12.653072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38b6ed-25bb-437d-96d6-4ef724fac986 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:12.653072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38b6ed-25bb-437d-96d6-4ef724fac986 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:12.653575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38b6ed-25bb-437d-96d6-4ef724fac986 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:12.654809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 697/1368] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:38:12.659501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3046b-478f-48ae-aba5-1ba58ebf30fe None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:12.659788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3046b-478f-48ae-aba5-1ba58ebf30fe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:12.660067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3046b-478f-48ae-aba5-1ba58ebf30fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:12.664217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:12.665816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951db23-5c76-45cd-8dd8-ac6f400c8073 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:12.685723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 672/1369] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:12.738457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bfd3046b-478f-48ae-aba5-1ba58ebf30fe None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:12.739818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bfd3046b-478f-48ae-aba5-1ba58ebf30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:12.854654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:12.866099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 698/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:13.203215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:13.203215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:13.203215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:13.206371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 673/1371] 10.222.0.60 () {54 vars in 964 bytes} [Fri Sep 9 11:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:13.209308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea9a20-7928-4ea4-b750-42d79ec92ee0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:13.209597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea9a20-7928-4ea4-b750-42d79ec92ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:13.209907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea9a20-7928-4ea4-b750-42d79ec92ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:13.291191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-34ea9a20-7928-4ea4-b750-42d79ec92ee0 None None] MFA Rules not processed for user `2eccd6d01b9a48cf946ea37543cf8c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:13.292394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34ea9a20-7928-4ea4-b750-42d79ec92ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:13.382646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 699/1372] 10.222.0.60 () {58 vars in 1061 bytes} [Fri Sep 9 11:38:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:13.388389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:13.389877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:13.421506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:13.502447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639dcb1-4794-48d5-99aa-1112898b9303 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:13.582727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:13.589605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:13.589605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:13.589605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:13.589605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:13.671316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:13.709248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:13.710760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd94eaf-31f5-42e0-8983-e75d40d22293 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:13.730420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 674/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:15.106851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:15.107416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:15.107416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:15.108787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 700/1374] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:15.114281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bdd-55b7-49d3-9e7a-a026572bcf01 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:15.114680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bdd-55b7-49d3-9e7a-a026572bcf01 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:15.114870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a14bdd-55b7-49d3-9e7a-a026572bcf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:15.182314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c7a14bdd-55b7-49d3-9e7a-a026572bcf01 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:15.183544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c7a14bdd-55b7-49d3-9e7a-a026572bcf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:15.285864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 675/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:15.570358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:15.571909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:15.658218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:15.721639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee3b6-76ba-478c-95e0-05cf9dc61450 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:15.841201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:15.841564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:15.841714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:15.841979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:15.843637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:15.896335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:15.940157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:15.941735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c53f4a-9280-443d-bf42-142e0422f439 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:15.960766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 701/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:18.315926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:18.316827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:18.317272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:18.318800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 676/1377] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:18.324801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f466d-7ee8-459c-ba6f-6d365cab5fc0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:18.325202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f466d-7ee8-459c-ba6f-6d365cab5fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:18.325368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f466d-7ee8-459c-ba6f-6d365cab5fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:18.398465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-442f466d-7ee8-459c-ba6f-6d365cab5fc0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:18.399680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-442f466d-7ee8-459c-ba6f-6d365cab5fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:18.510092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 702/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:18.797030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:18.798691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:18.837967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:18.951799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdd6f1-24ce-4846-92ca-911a61949eb9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:19.071212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:19.077352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:19.078091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:19.078474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:19.078847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:19.080768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:19.183586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:19.185418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6fb3c-1d20-4fb3-a7c0-f30880441067 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:19.225964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 677/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:21.591459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:21.591988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:21.592222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:21.593923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 703/1380] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:21.599258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e49c8-cc3d-4105-be3e-4098e45d3d78 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:21.599668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e49c8-cc3d-4105-be3e-4098e45d3d78 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:21.599845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e49c8-cc3d-4105-be3e-4098e45d3d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:21.669305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f26e49c8-cc3d-4105-be3e-4098e45d3d78 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:21.670615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f26e49c8-cc3d-4105-be3e-4098e45d3d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:21.739908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:21.773808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 678/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:22.080471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:22.082543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:22.119939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:22.257915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee1bb-d17c-4a4d-b02b-6a1a441db94a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:22.354228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:22.370354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:22.371273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:22.371273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:22.371508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:22.373091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:22.467397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:22.468999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dadb6-9d5d-475c-8715-1725cad96066 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:22.490970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 704/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:24.781402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:24.782125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:24.782125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:24.783306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 679/1383] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:24.794375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b6c81-0db3-43e1-9a77-baaf03e8eac9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:24.794890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b6c81-0db3-43e1-9a77-baaf03e8eac9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:24.795059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b6c81-0db3-43e1-9a77-baaf03e8eac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:24.871938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c15b6c81-0db3-43e1-9a77-baaf03e8eac9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:24.873159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c15b6c81-0db3-43e1-9a77-baaf03e8eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:24.936543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:24.985585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 705/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:25.281757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:25.283409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:25.454161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54498e4-1d8f-4416-a539-e6c34ce18872 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:25.510245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:25.580367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc647f40-5921-41af-b655-429524e02420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:25.580939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647f40-5921-41af-b655-429524e02420 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:25.581236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647f40-5921-41af-b655-429524e02420 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:25.581535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647f40-5921-41af-b655-429524e02420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:25.583304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bc647f40-5921-41af-b655-429524e02420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:25.682915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647f40-5921-41af-b655-429524e02420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:25.684538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647f40-5921-41af-b655-429524e02420 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:25.703657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 680/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:27.875223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:27.875796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:27.876076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:27.877428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 706/1386] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:27.882779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07329b-bd03-4c1e-851b-0cc9ca2afef2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:27.883209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07329b-bd03-4c1e-851b-0cc9ca2afef2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:27.883339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07329b-bd03-4c1e-851b-0cc9ca2afef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:27.949897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fb07329b-bd03-4c1e-851b-0cc9ca2afef2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:27.951125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fb07329b-bd03-4c1e-851b-0cc9ca2afef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:27.959355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:28.047464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 681/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:28.340013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:28.341525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:28.493052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2013b70-4ed7-4c91-bd60-d8ea8e7c132a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:28.542648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:28.610681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:28.611288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:28.611565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:28.611835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:28.613457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:28.705174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:28.706854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d216fea-c462-43c4-bc70-030c95ae6fe0 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:28.725080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 707/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:29.835312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad0595-6ddf-4afa-b5a8-b69cf74bfc29 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:29.835919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad0595-6ddf-4afa-b5a8-b69cf74bfc29 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:29.835919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad0595-6ddf-4afa-b5a8-b69cf74bfc29 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:29.837179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 682/1389] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:29.842607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc9f3-a12e-4ff3-abc4-d00a5a350304 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:29.843065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc9f3-a12e-4ff3-abc4-d00a5a350304 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:29.843170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc9f3-a12e-4ff3-abc4-d00a5a350304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:29.906387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:29.908932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8f0bc9f3-a12e-4ff3-abc4-d00a5a350304 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:29.910155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8f0bc9f3-a12e-4ff3-abc4-d00a5a350304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:30.005738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 708/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:30.307100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:30.307385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:30.307662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:30.309022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 683/1391] 10.222.0.60 () {54 vars in 966 bytes} [Fri Sep 9 11:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:30.314497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32509-9395-4c2b-99f9-2e0b0df57a95 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:30.314975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32509-9395-4c2b-99f9-2e0b0df57a95 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:30.315250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32509-9395-4c2b-99f9-2e0b0df57a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:30.395310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8fd32509-9395-4c2b-99f9-2e0b0df57a95 None None] MFA Rules not processed for user `10d651db5bf74166bff093d317d83a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:30.396557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8fd32509-9395-4c2b-99f9-2e0b0df57a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:30.467271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:30.472913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 709/1392] 10.222.0.60 () {58 vars in 1063 bytes} [Fri Sep 9 11:38:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:30.482070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:30.483805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:30.601168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee480dba-cad2-42ca-a92f-970fc184242d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:30.638241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:30.676128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:30.676726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:30.676997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:30.677258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:30.679380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:30.786644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:30.788594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8158a-0834-4dc2-9cf4-fa1fa90a749a service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:30.807563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 684/1393] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:31.129715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.133620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.133620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.133620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 710/1394] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:38:31.137994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57968d-eabc-4567-a65e-23b4f187d8df None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.138482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57968d-eabc-4567-a65e-23b4f187d8df None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.138588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57968d-eabc-4567-a65e-23b4f187d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.205230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1c57968d-eabc-4567-a65e-23b4f187d8df None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:31.206561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c57968d-eabc-4567-a65e-23b4f187d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:31.225930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:31.313070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 685/1395] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:31.609981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:31.611674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:31.679858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f73-2208-460f-84b5-a9a30c8c7cd5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.680154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f73-2208-460f-84b5-a9a30c8c7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.680434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac1f73-2208-460f-84b5-a9a30c8c7cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.747242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3eac1f73-2208-460f-84b5-a9a30c8c7cd5 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:31.748451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3eac1f73-2208-460f-84b5-a9a30c8c7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:31.766082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bdd51f-9608-42da-9fb1-841e37011ad4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:31.835851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 686/1396] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:31.837757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:31.883454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444fdd-abfe-47a4-9f5b-5fef23361ec3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.883740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444fdd-abfe-47a4-9f5b-5fef23361ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.884011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444fdd-abfe-47a4-9f5b-5fef23361ec3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.885357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 687/1397] 10.222.0.60 () {54 vars in 943 bytes} [Fri Sep 9 11:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:38:31.888486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:31.889187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.889528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.889807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.891648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd089b-b910-4373-9e6b-2688609682df None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:31.891780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:31.891928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd089b-b910-4373-9e6b-2688609682df None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:31.892197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd089b-b910-4373-9e6b-2688609682df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:31.908431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:31.972755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4cdd089b-b910-4373-9e6b-2688609682df None None] MFA Rules not processed for user `da9b976e48d3468b82d9168efced0b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:31.973989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cdd089b-b910-4373-9e6b-2688609682df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:31.984039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:31.985613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:32.004717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 711/1398] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:32.081862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 688/1399] 10.222.0.60 () {58 vars in 1040 bytes} [Fri Sep 9 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:32.091393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:32.092640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:32.179196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:32.183024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574bfbf-cc72-4400-97a2-1e22282ab3c9 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:32.295391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ade8e-7671-49af-8234-305789af18af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:32.296023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ade8e-7671-49af-8234-305789af18af service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:32.296305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ade8e-7671-49af-8234-305789af18af service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:32.296573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ade8e-7671-49af-8234-305789af18af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:32.298360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e55ade8e-7671-49af-8234-305789af18af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:32.391543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ade8e-7671-49af-8234-305789af18af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:32.393119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ade8e-7671-49af-8234-305789af18af service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:32.404824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 712/1400] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:38:34.383842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:34.384488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:34.384488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:34.385739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 689/1401] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:34.391150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933db-0b52-4afc-bffa-ae91cdf051c4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:34.391646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933db-0b52-4afc-bffa-ae91cdf051c4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:34.391778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933db-0b52-4afc-bffa-ae91cdf051c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:34.431733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:34.472842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2f7933db-0b52-4afc-bffa-ae91cdf051c4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:34.475000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2f7933db-0b52-4afc-bffa-ae91cdf051c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:34.598102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 713/1402] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:34.889450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:34.891023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:34.977747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:35.062713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb733-5642-4c4a-8243-b580fedf9632 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:35.193782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:35.194520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:35.194866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:35.195225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:35.196901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:35.238879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:35.304649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:35.306328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2154d4-fb85-4a46-9d6f-d734634a5f9d service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:35.326158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 690/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:37.679906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:37.680478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:37.680478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:37.685608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 714/1404] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:37.687188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cb51-77b4-470d-a2ee-5aea0e54c955 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:37.687635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cb51-77b4-470d-a2ee-5aea0e54c955 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:37.687726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cb51-77b4-470d-a2ee-5aea0e54c955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:37.758544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3007cb51-77b4-470d-a2ee-5aea0e54c955 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:37.759949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3007cb51-77b4-470d-a2ee-5aea0e54c955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:37.868890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 691/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:38:38.159276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:38.160750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:38.281609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:38.328559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504f3be-cd8d-4b88-b4cd-81aedf69c983 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:38.450644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:38.451264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:38.451608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:38.451937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:38.453576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:38.537980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:38.557625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:38.559471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97250a8-405a-4af7-ab3e-ab371a93df96 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:38.580734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 715/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:40.980416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:40.980961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:40.980961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:40.982209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 692/1407] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:40.987773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863bb4-e95a-4e8c-9de5-25e81574a76f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:40.988256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863bb4-e95a-4e8c-9de5-25e81574a76f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:40.988343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b863bb4-e95a-4e8c-9de5-25e81574a76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:41.058500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2b863bb4-e95a-4e8c-9de5-25e81574a76f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:41.059725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2b863bb4-e95a-4e8c-9de5-25e81574a76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:41.170524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 716/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:41.471155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:38:41.472707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:41.483865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:41.641676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620c3d0-7b67-442a-932d-5704e0727ec3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:38:41.737651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:41.774563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:38:41.775247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:41.775543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:41.775840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:41.777882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:41.883114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:38:41.884878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883f03a-8351-40be-9115-3be89952dc44 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:38:41.904448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 693/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:38:51.168973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=79 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:51.901936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=58 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:56.723666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd666b-13e6-4c98-9dcc-ed39e23a20ed None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:56.724227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd666b-13e6-4c98-9dcc-ed39e23a20ed None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:56.724227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd666b-13e6-4c98-9dcc-ed39e23a20ed None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:56.725551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 717/1410] 10.222.0.60 () {54 vars in 918 bytes} [Fri Sep 9 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:56.730595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac0f0-b960-4025-a0f6-0933ef490f5a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:56.731044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac0f0-b960-4025-a0f6-0933ef490f5a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:56.731143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ac0f0-b960-4025-a0f6-0933ef490f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:56.805638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8d0ac0f0-b960-4025-a0f6-0933ef490f5a None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:56.806894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8d0ac0f0-b960-4025-a0f6-0933ef490f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:56.911716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 694/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Sep 9 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:56.927094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5fd21-a0a4-4618-b796-fe24f0706fc2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:56.927316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5fd21-a0a4-4618-b796-fe24f0706fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:56.927589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5fd21-a0a4-4618-b796-fe24f0706fc2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:56.928941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 718/1412] 10.222.0.60 () {54 vars in 918 bytes} [Fri Sep 9 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:56.933495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd122f1-1f16-4091-b213-5cf9f521cb74 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:56.933780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd122f1-1f16-4091-b213-5cf9f521cb74 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:56.934087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd122f1-1f16-4091-b213-5cf9f521cb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:57.006901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:57.009882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5cd122f1-1f16-4091-b213-5cf9f521cb74 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:57.011128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5cd122f1-1f16-4091-b213-5cf9f521cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:57.126556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 695/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Sep 9 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:38:59.768834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:59.769354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:59.769354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:59.770958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:38:59.776342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5b4c-adaa-48d5-aaa8-2a845323fb46 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:38:59.776799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5b4c-adaa-48d5-aaa8-2a845323fb46 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:38:59.776948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5b4c-adaa-48d5-aaa8-2a845323fb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:38:59.846158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2bbb5b4c-adaa-48d5-aaa8-2a845323fb46 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:38:59.847386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2bbb5b4c-adaa-48d5-aaa8-2a845323fb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:38:59.902220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:38:59.943955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:39:00.237566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:00.239753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:00.386100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e128-d084-49b1-9bb3-3b6bbea488d0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:00.458626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:00.497293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:00.497825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:00.498128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:00.498398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:00.499991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:00.589375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:00.590993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b507fa1-d185-4c8e-b1fc-23e2d97d3b3f service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:00.608886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 720/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:01.753834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:01.754410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:01.754410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:01.755750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:01.761987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d066fb-5bc4-4f72-9d2c-947ba7f0d0f6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:01.762439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d066fb-5bc4-4f72-9d2c-947ba7f0d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:01.762555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d066fb-5bc4-4f72-9d2c-947ba7f0d0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:01.838560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-20d066fb-5bc4-4f72-9d2c-947ba7f0d0f6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:01.839744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20d066fb-5bc4-4f72-9d2c-947ba7f0d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:01.866663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:01.935207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:02.087126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:02.088550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:02.233191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ef4c-54de-4f83-a7a4-a10bd783313b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:02.273612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:02.345816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:02.346375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:02.346656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:02.346930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:02.348592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:02.438215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:02.439824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80952204-532b-42c8-ba62-baa2fe84018e service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:02.458757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 698/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:03.583034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:03.583611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:03.583611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:03.584919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:03.591128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d2c6-801d-44bc-8f11-953abec7386c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:03.591540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d2c6-801d-44bc-8f11-953abec7386c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:03.591675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d2c6-801d-44bc-8f11-953abec7386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:03.644721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:03.665543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b6f6d2c6-801d-44bc-8f11-953abec7386c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:03.666802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b6f6d2c6-801d-44bc-8f11-953abec7386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:03.760959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:03.915957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:03.917478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:04.062457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951cae92-90cd-427d-ad3c-3e8a58c95c49 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:04.074742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:04.174358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:04.174884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:04.175162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:04.175430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:04.177034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:04.265206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:04.266804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baacc8-acc5-4d73-8ee4-91679867d62f service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:04.284377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 723/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:05.396803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:05.397257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:05.397367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:05.398719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:05.405128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022d3c9-045a-4711-aa86-52cdc6e2f46e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:05.405586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022d3c9-045a-4711-aa86-52cdc6e2f46e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:05.405685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022d3c9-045a-4711-aa86-52cdc6e2f46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:05.425670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:05.482620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8022d3c9-045a-4711-aa86-52cdc6e2f46e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:05.483828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8022d3c9-045a-4711-aa86-52cdc6e2f46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:05.577762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:05.729168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:05.730746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:05.839513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:05.876506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cc9b7-e285-49d8-ae0c-b38fec58f9ac None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:05.990557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:05.991086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:05.991366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:05.991640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:05.993268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:06.072546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:06.089651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:06.114878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef658bb-7cf6-4c36-b414-ce0088202d6f service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:06.114878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 701/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:07.236579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:07.237157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:07.237157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:07.238697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:07.244831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3064-974f-4c19-9731-b974e9d91923 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:07.245285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3064-974f-4c19-9731-b974e9d91923 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:07.245392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b3064-974f-4c19-9731-b974e9d91923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:07.319371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6b8b3064-974f-4c19-9731-b974e9d91923 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:07.320577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6b8b3064-974f-4c19-9731-b974e9d91923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:07.419365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:07.575301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:07.576736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:07.658581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:07.721044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb957-514f-4633-94b2-4c9cc7ed280d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:07.836542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:07.837071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:07.837348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:07.837616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:07.839580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:07.887507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:07.928472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:07.930560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d76bd1-d1fd-428d-98e4-13df6bce0427 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:07.949052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 726/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:09.108113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:09.108656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:09.108656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:09.109955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:09.116227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5cefc-4e16-4836-b94f-dd8e3a54b418 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:09.116646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5cefc-4e16-4836-b94f-dd8e3a54b418 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:09.116768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5cefc-4e16-4836-b94f-dd8e3a54b418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:09.192557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f4c5cefc-4e16-4836-b94f-dd8e3a54b418 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:09.193824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f4c5cefc-4e16-4836-b94f-dd8e3a54b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:09.289278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:39:09.440231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:09.442470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:09.491025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:09.585355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258ac2b-bcf7-4731-b5c0-27caaa504fa0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:09.695151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:09.695673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:09.695947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:09.696213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:09.698038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:09.704704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:09.786253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:09.787855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1f086-455a-4817-a472-1117442fc242 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:09.805901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 704/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:10.919037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:10.919602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:10.919602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:10.920917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:10.927204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee0fe7-7177-491c-9196-8ddbd9a6e99a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:10.927627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee0fe7-7177-491c-9196-8ddbd9a6e99a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:10.927728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee0fe7-7177-491c-9196-8ddbd9a6e99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:11.003689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-abee0fe7-7177-491c-9196-8ddbd9a6e99a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:11.004820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-abee0fe7-7177-491c-9196-8ddbd9a6e99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:11.074403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:11.103876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:11.254338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:11.255781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:11.273524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:11.400728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ca733-4949-4967-a31f-42a75fe69980 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:11.486238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:11.513272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:11.513272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:11.513272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:11.513272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:11.513834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:11.602767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:11.604341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b46e2-f94a-44b8-9cb0-b8167eca28ca service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:11.622504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 729/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:12.744543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:12.745190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:12.745190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:12.746487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:12.755138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6967-833f-4741-b30f-21f010334ca1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:12.755595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6967-833f-4741-b30f-21f010334ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:12.755708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6967-833f-4741-b30f-21f010334ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:12.944758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0c6e6967-833f-4741-b30f-21f010334ca1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:12.945883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0c6e6967-833f-4741-b30f-21f010334ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:12.986888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:13.045860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:13.201852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:13.203372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:13.345879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f1bcd-4078-4b30-9e1b-40deab12b2e2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:13.405260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:13.471231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:13.471744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:13.472650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:13.472917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:13.474532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:13.561544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:13.563174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6c918-8131-4ddf-89e8-0d379e070c7d service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:13.581393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 707/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:14.683772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:14.684296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:14.684296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:14.685556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:14.691994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6568df-3440-45c0-b49a-32d5e7e3cf82 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:14.692394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6568df-3440-45c0-b49a-32d5e7e3cf82 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:14.692589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6568df-3440-45c0-b49a-32d5e7e3cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:14.767286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ac6568df-3440-45c0-b49a-32d5e7e3cf82 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:14.768491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ac6568df-3440-45c0-b49a-32d5e7e3cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:14.771006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:14.864343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:15.015257 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:15.016843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:15.161159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2371b1a-d670-4d20-ad76-86a286f5e335 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:15.187747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:15.271470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:15.272002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:15.272285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:15.272552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:15.274160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:15.365034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:15.366640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5537521-1d4d-476b-98ad-1b523952ee57 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:15.384303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 732/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:17.470287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:17.470846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:17.470846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:17.472692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:39:17.478099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18221593-adeb-412a-aed1-c0d053fbd024 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:17.478509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18221593-adeb-412a-aed1-c0d053fbd024 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:17.478685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18221593-adeb-412a-aed1-c0d053fbd024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:17.519547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:17.544213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-18221593-adeb-412a-aed1-c0d053fbd024 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:17.545420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-18221593-adeb-412a-aed1-c0d053fbd024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:17.640786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:17.928274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:17.929803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:18.031925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a83981-bfd7-4b74-801e-abe0b3e9af40 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:18.063056 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:18.105202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:18.105824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:18.106132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:18.106403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:18.107988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:18.213861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:18.215253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80925cb-a4f2-4a2b-bcd5-66ef33ea90b5 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:18.232639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 710/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:20.515121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:20.515600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:20.515720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:20.516970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:20.522765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1988-7418-4d2b-aae2-c569e637ecfb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:20.523200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1988-7418-4d2b-aae2-c569e637ecfb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:20.533670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f1988-7418-4d2b-aae2-c569e637ecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:20.596337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7f1f1988-7418-4d2b-aae2-c569e637ecfb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:20.598134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7f1f1988-7418-4d2b-aae2-c569e637ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:20.610751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:20.694184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:20.980172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:20.981755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:21.084414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d30929-e8ab-4025-94df-e10f3787a9cd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:21.097083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:21.158794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:21.159454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:21.159731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:21.160003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:21.161602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:21.271056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:21.272384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3157-290c-4cf1-94dc-133dddb5e889 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:21.290334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 735/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:23.826965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:23.827499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:23.827499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:23.828759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:23.834184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56b695-e1d3-4310-9218-d56969186613 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:23.834646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56b695-e1d3-4310-9218-d56969186613 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:23.834754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56b695-e1d3-4310-9218-d56969186613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:23.882168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:23.899383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cd56b695-e1d3-4310-9218-d56969186613 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:23.900621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cd56b695-e1d3-4310-9218-d56969186613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:23.996890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:24.283507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:24.285814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:24.389241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c777b7-5d3b-4359-bf1c-49cce48d8d58 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:24.428954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:24.462030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:24.462661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:24.462940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:24.463215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:24.464827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:24.572573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:24.573946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a61159-9f4b-46cf-ac16-cde0865ad90e service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:24.592033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 713/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:26.859596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:26.860172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:26.860172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:26.861440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:26.867429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a350-dd5a-4f43-9c49-f6ef05e9e00f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:26.867847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a350-dd5a-4f43-9c49-f6ef05e9e00f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:26.867979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a350-dd5a-4f43-9c49-f6ef05e9e00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:26.932865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-98a9a350-dd5a-4f43-9c49-f6ef05e9e00f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:26.934114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98a9a350-dd5a-4f43-9c49-f6ef05e9e00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:26.955171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:27.029317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:27.315892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:27.317460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:27.420739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d336993-66cd-4a9a-8f4c-e277245cc981 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:27.440580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:27.493029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:27.493633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:27.493910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:27.494229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:27.495819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:27.603133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:27.604475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1e747-58d7-408b-ba90-59fb9b9cfe52 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:27.621965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 738/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:39:29.866217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:29.866792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:29.866792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:29.868130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:29.873356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436e20-3055-415b-be32-c149d7e09840 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:29.879299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436e20-3055-415b-be32-c149d7e09840 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:29.879299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436e20-3055-415b-be32-c149d7e09840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:29.937426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:29.940567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-99436e20-3055-415b-be32-c149d7e09840 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:29.941890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-99436e20-3055-415b-be32-c149d7e09840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:30.036980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:30.325847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:30.327431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:30.429857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98b295-7263-4dd5-bb25-37d45165b17a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:30.479687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:30.502905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:30.503516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:30.503795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:30.504071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:30.505695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:30.619170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:30.620601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986745e-6d03-4592-b861-b397e911171e service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:30.638953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 716/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:32.840590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:32.841144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:32.841144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:32.842492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:32.847860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d08c3-0fa4-4306-ab7c-cd96e74589c6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:32.848273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d08c3-0fa4-4306-ab7c-cd96e74589c6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:32.848417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d08c3-0fa4-4306-ab7c-cd96e74589c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:32.914872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c19d08c3-0fa4-4306-ab7c-cd96e74589c6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:32.916092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c19d08c3-0fa4-4306-ab7c-cd96e74589c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:32.945261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:33.012773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:39:33.299218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:33.300762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:33.404939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04bfef-374d-4972-934d-0f895fdb07b9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:33.434504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:33.480631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:33.481233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:33.481610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:33.481820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:33.483760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:33.594274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:33.595640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6065a0-2919-40a0-8ac1-3a1b3c37afd5 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:33.614151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 741/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:35.823673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:35.824201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:35.824201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:35.825497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:35.831251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca07f-3d01-4ca7-8052-fa47e9de9f93 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:35.831672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca07f-3d01-4ca7-8052-fa47e9de9f93 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:35.831838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca07f-3d01-4ca7-8052-fa47e9de9f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:35.898475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b54ca07f-3d01-4ca7-8052-fa47e9de9f93 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:35.899686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b54ca07f-3d01-4ca7-8052-fa47e9de9f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:35.908675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:35.995338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:39:36.286028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:36.287636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:36.393397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eebd09-c117-424d-a61f-a64cd218f06d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:36.449316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:36.467642 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:36.468246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:36.468518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:36.472881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:36.472881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:36.581696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:36.583075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5b394-358f-49b8-8457-e689ffe7fa9f service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:36.718868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 719/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:38.940718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:38.941328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:38.941328 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:38.943212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:39:38.949140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fddce-80ce-4f0f-9b21-8eec95edff0e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:38.949674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fddce-80ce-4f0f-9b21-8eec95edff0e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:38.949674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fddce-80ce-4f0f-9b21-8eec95edff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:39.029393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-935fddce-80ce-4f0f-9b21-8eec95edff0e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:39.030593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-935fddce-80ce-4f0f-9b21-8eec95edff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:39.075359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:39.136800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:39.441391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:39.442968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:39.552754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0090581-e497-4faf-b80e-e0b827ed162e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:39.589936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:39.625197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:39.625784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:39.626081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:39.626347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:39.627943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:39.735057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:39.736438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0396d34-999b-4874-badf-4c116fc14718 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:39.768434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 744/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:42.055856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:42.056441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:42.056441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:42.057775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:42.063382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121243bb-36f1-49f6-8c5f-395f27b2774c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:42.064326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121243bb-36f1-49f6-8c5f-395f27b2774c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:42.064444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121243bb-36f1-49f6-8c5f-395f27b2774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:42.130607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-121243bb-36f1-49f6-8c5f-395f27b2774c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:42.131821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-121243bb-36f1-49f6-8c5f-395f27b2774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:42.149912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:42.227176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:42.508442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:42.509969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:42.610654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c961a1-9a4f-4819-af8f-fb5b9df00b13 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:42.675339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:42.681625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:42.682277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:42.691896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:42.691896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:42.691896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:42.801015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:42.802482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f62789-a3a1-4d31-b7f3-ad1893944c0a service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:42.820437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 722/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:45.010844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:45.011367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:45.011367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:45.012638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:39:45.018710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3272-bb4c-461d-9dbe-dbc0e68c4224 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:45.019128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3272-bb4c-461d-9dbe-dbc0e68c4224 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:45.019243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3272-bb4c-461d-9dbe-dbc0e68c4224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:45.084159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6c6c3272-bb4c-461d-9dbe-dbc0e68c4224 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:39:45.085378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6c6c3272-bb4c-461d-9dbe-dbc0e68c4224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:45.132259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:45.179230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:39:45.460186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:39:45.461714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:45.566497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c15419-54e1-4869-bc5a-abb760572b59 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:39:45.613569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:45.638637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:39:45.639178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:39:45.639447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:39:45.639712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:39:45.641279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:39:45.751191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:39:45.752604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfd495-5245-4560-af6a-ff6c692bab7e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:39:45.770568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 747/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:39:55.177719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=22 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:39:55.768271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=66 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:06.542317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdf95-9120-4820-9c54-faafcea2bf93 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:06.542880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdf95-9120-4820-9c54-faafcea2bf93 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:06.542880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fdf95-9120-4820-9c54-faafcea2bf93 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:06.544299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 724/1471] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:06.550246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c284-82ed-42ae-a4aa-08aa3c6b1be6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:06.550646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c284-82ed-42ae-a4aa-08aa3c6b1be6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:06.550793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53c284-82ed-42ae-a4aa-08aa3c6b1be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:06.615322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2e53c284-82ed-42ae-a4aa-08aa3c6b1be6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:06.616505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2e53c284-82ed-42ae-a4aa-08aa3c6b1be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:06.710254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 748/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:07.014406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:07.014691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:07.014961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:07.016331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 725/1473] 10.222.0.60 () {54 vars in 966 bytes} [Fri Sep 9 11:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:07.021207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce351281-2805-44bd-845f-c053b7dcb01c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:07.021494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce351281-2805-44bd-845f-c053b7dcb01c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:07.021770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce351281-2805-44bd-845f-c053b7dcb01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:07.097635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ce351281-2805-44bd-845f-c053b7dcb01c None None] MFA Rules not processed for user `10d651db5bf74166bff093d317d83a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:07.098867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ce351281-2805-44bd-845f-c053b7dcb01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:07.118709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:07.173886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 749/1474] 10.222.0.60 () {58 vars in 1063 bytes} [Fri Sep 9 11:40:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:07.181641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:07.183177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:07.284973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7154ab-c141-4c65-9cd0-45e19ce257cd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:07.356377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:07.356971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:07.357244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:07.357511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:07.359696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:07.414571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:07.470281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:07.471725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e8700-fddf-477d-b09f-f5ebd591ddbb service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:07.491301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 726/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:09.750249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:09.750800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:09.750800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:09.752062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 750/1476] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:09.757222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707703-36ec-46ae-bb81-596fb4dcd17a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:09.757624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707703-36ec-46ae-bb81-596fb4dcd17a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:09.757766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707703-36ec-46ae-bb81-596fb4dcd17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:09.821726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-60707703-36ec-46ae-bb81-596fb4dcd17a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:09.822948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-60707703-36ec-46ae-bb81-596fb4dcd17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:09.917397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 727/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:10.200844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:10.202441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:10.326900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da280b6-8e6c-4fe8-83a1-17b20482d3d7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:10.376459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:10.419155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:10.419753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:10.420080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:10.420349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:10.420978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:10.423251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:40:10.423251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:40:10.423251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:40:10.424368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54253-e119-468e-a759-82c83884bdf8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:10.443305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 751/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Sep 9 11:40:10 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:40:12.603829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:12.604394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:12.604394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:12.605708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 728/1479] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:40:12.611264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1d73-b92e-44e7-bdd5-f687748c4a71 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:12.611675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1d73-b92e-44e7-bdd5-f687748c4a71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:12.611820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1d73-b92e-44e7-bdd5-f687748c4a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:12.678145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-143b1d73-b92e-44e7-bdd5-f687748c4a71 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:12.679314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-143b1d73-b92e-44e7-bdd5-f687748c4a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:12.774786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 752/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:13.074006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:13.075544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:13.085689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:13.219019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54188f0-c314-4c1b-b609-ca65aa6a97a6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:13.294781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:13.326957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:13.328001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:13.328274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:13.328540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:13.330133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:13.418800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:13.420370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be0b9-a4b6-495f-b0d0-85d6dd9fb381 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:13.437589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 729/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:15.767790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:15.768324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:15.768324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:15.769579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 753/1482] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:15.774969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7381b6-d7ac-4789-a6d1-a32d76642768 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:15.775405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7381b6-d7ac-4789-a6d1-a32d76642768 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:15.775512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7381b6-d7ac-4789-a6d1-a32d76642768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:15.840296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8d7381b6-d7ac-4789-a6d1-a32d76642768 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:15.841501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8d7381b6-d7ac-4789-a6d1-a32d76642768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:15.887248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:15.940605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 730/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:16.223306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:16.224879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:16.248669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:16.326033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950d32d-dbf2-45e6-8a75-ed6e0f5d73b8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:16.401728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:16.402372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:16.402653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:16.402921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:16.404522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:16.478883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:16.513754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:16.516336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dda4f8-6d9f-4e70-bfce-2109772807b9 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:16.534421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 754/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:18.731137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:18.731899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:18.732202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:18.734189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 731/1485] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:18.739365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da331639-4247-4f85-9387-a339438d10c6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:18.739781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da331639-4247-4f85-9387-a339438d10c6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:18.739914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da331639-4247-4f85-9387-a339438d10c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:18.805416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-da331639-4247-4f85-9387-a339438d10c6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:18.806622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-da331639-4247-4f85-9387-a339438d10c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:18.901475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 755/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:19.188978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:19.190550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:19.291897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672a982-517f-4ac0-b128-8930f8ac4c25 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:19.362013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:19.363458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:19.364061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:19.364337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:19.364608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:19.366252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:19.473304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:19.474697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489b0f7-5fe8-4268-adbf-909452cf5c64 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:19.492481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 732/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:21.729528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:21.730112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:21.730112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:21.731362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 756/1488] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:21.737302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0ac6-c252-434e-b670-83f138372b3a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:21.737693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0ac6-c252-434e-b670-83f138372b3a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:21.737846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0ac6-c252-434e-b670-83f138372b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:21.801964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6f4f0ac6-c252-434e-b670-83f138372b3a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:21.803191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6f4f0ac6-c252-434e-b670-83f138372b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:21.855124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:21.898556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 733/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:40:22.194992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:22.196575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:22.232694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:22.298454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124dd1d-94dd-4c83-ba57-1faefdc0a5a2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:22.372561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:22.373158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:22.373431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:22.374318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:22.375907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:22.453680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:22.482107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:22.484451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab399ae-8856-462c-8a0f-c71cf5bbf123 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:22.502453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 757/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:25.042847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:25.043399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:25.043399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:25.044689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 734/1491] 10.222.0.60 () {54 vars in 927 bytes} [Fri Sep 9 11:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:40:25.050755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d87c7-b790-4e8f-8530-59080129f463 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:25.051281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d87c7-b790-4e8f-8530-59080129f463 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:25.051281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d87c7-b790-4e8f-8530-59080129f463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:25.120476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f86d87c7-b790-4e8f-8530-59080129f463 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:40:25.121658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f86d87c7-b790-4e8f-8530-59080129f463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:25.216780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 758/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Sep 9 11:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:40:25.507500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:40:25.509101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:25.610476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af6f06-f5fd-445c-8d83-f4419e803c41 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:40:25.681390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:40:25.681951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:40:25.682275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:40:25.682544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:40:25.684129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:40:25.692936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:25.790957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:40:25.792381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57814b7-8eb4-4a94-ac44-dd7aa191ca2b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:40:25.810187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 735/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:40:35.215234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=82 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:40:35.807923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=50 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:55.059173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:55.059748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:55.059748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:55.135755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:41:55.136953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:55.232344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 759/1494] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:41:55.240308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:55.240816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:55.240928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:55.314952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:41:55.316375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:55.411550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 736/1495] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:41:55.416813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:55.418011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:55.429317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d5e84-ac53-4592-8495-fbbcd1dbb886 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:55.489757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:55.522934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:55.523530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:55.523802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:55.524067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:55.524703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:41:55.525515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:55.526525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:55.526525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:41:55.526525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:41:55.527397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:55.538315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 760/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:41:55.547202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:55.556298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a4b46-a1da-4489-a501-a961aa54c8f3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:55.616203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:55.647851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:55.648440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:55.648712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:55.648974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:55.649591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:55.651852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:55.651852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:41:55.651852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:41:55.653036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:55.763436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 737/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:41:55 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:41:55.767713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:55.769263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:55.802285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410a4d7-9786-445b-8da6-f068b18ba66e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:55.894955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:55.895589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:55.895869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:55.896144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:55.896776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:55.897766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:55.897766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:41:55.897766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:41:55.898983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:56.027310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 761/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:41:55 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:41:56.031291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:56.032807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.094193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79263319-fe0a-4836-b598-b5240ec0c055 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:56.175540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:56.185675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:56.186984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.186984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.186984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.187430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:56.188429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:56.188429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:41:56.188429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:41:56.189300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:56.202678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 738/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:41:56.206796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:56.208305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.219772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a503d7-b8b3-48d2-aded-0aae3400b912 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:56.245159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:56.314077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:56.314731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.315014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.315284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.315931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:56.317223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:56.317223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:41:56.317223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:41:56.318133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251e2a1-d909-4d48-85a2-edff88f3997c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:56.330927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 762/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:41:56.335034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:56.336566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.370397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085806-07a2-4a1f-a267-0bb877a27f68 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:56.464598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:56.465183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.465460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.465730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] PATH_INFO: `/v3/projects/c2debd4dad5348a591cca81bf1ad96e4/users/e8ff56e0232443d4abc8036659fa031c/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.488045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2debd4dad5348a591cca81bf1ad96e4, user_id=e8ff56e0232443d4abc8036659fa031c, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:56.494704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:56.494704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:41:56.494704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:41:56.494704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc04d0-4e9c-45f7-8555-01db0b80e0f1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:56.601203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 739/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:41:56 2022] PUT /identity/v3/projects/c2debd4dad5348a591cca81bf1ad96e4/users/e8ff56e0232443d4abc8036659fa031c/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:41:56.614243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e066a-d8aa-45f2-8ad0-d212b7757cc4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.614559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e066a-d8aa-45f2-8ad0-d212b7757cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.614832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e066a-d8aa-45f2-8ad0-d212b7757cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.670912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ac2e066a-d8aa-45f2-8ad0-d212b7757cc4 None None] MFA Rules not processed for user `e8ff56e0232443d4abc8036659fa031c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:41:56.672122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ac2e066a-d8aa-45f2-8ad0-d212b7757cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.749671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 763/1502] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:41:56.762947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.763228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.763527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.764840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 740/1503] 10.222.0.60 () {54 vars in 964 bytes} [Fri Sep 9 11:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:41:56.769552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df9186-348b-4ed6-992b-caca98137b59 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:56.769844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df9186-348b-4ed6-992b-caca98137b59 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:56.770151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df9186-348b-4ed6-992b-caca98137b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:56.801151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:56.845648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-00df9186-348b-4ed6-992b-caca98137b59 None None] MFA Rules not processed for user `2eccd6d01b9a48cf946ea37543cf8c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:41:56.846861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-00df9186-348b-4ed6-992b-caca98137b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.919265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 764/1504] 10.222.0.60 () {58 vars in 1061 bytes} [Fri Sep 9 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:41:56.927050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:56.928530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:56.980535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:57.035083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a699414-137b-44d6-8427-1caea520ac50 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:57.106770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:57.107362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:57.107641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:57.107903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:57.109510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:57.209581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:57.211046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:57.214688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:57.228786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 741/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:41:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:41:57.274395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33453bc-c07a-4951-a487-caced1056252 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:57.274812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33453bc-c07a-4951-a487-caced1056252 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:57.274940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33453bc-c07a-4951-a487-caced1056252 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:57.276196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 765/1506] 10.222.0.60 () {52 vars in 830 bytes} [Fri Sep 9 11:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:41:57.279651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:57.280854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:57.378976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ed416-06f9-473d-8830-e6e6d5b41a64 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:57.470473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:57.471093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:57.471371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:57.471638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] PATH_INFO: `/v3/users/e8ff56e0232443d4abc8036659fa031c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:57.484651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ff56e0232443d4abc8036659fa031c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:57.485673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:57.485673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:41:57.485673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:41:57.486924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:57.496957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:57.685984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ff56e0232443d4abc8036659fa031c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:57.708576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ff56e0232443d4abc8036659fa031c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:57.723287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ff56e0232443d4abc8036659fa031c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:57.794301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ff56e0232443d4abc8036659fa031c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:57.794728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ff56e0232443d4abc8036659fa031c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:57.800404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 742/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:41:57 2022] DELETE /identity/v3/users/e8ff56e0232443d4abc8036659fa031c => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:41:57.804731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:57.805020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:57.805295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:57.868292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:41:57.869515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:57.943289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:57.955128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 766/1508] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:41:57.965444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:57.966677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:58.066144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df036-5950-4fc7-bc91-bde17e6d2f93 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:58.119035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:41:58.137938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:58.138576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:58.138853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:58.139124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:58.140729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:58.258424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:58.260284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fcd2a-502e-437d-905f-d681d4ceeea6 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:58.279080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 743/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:41:58.747974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:41:58.749608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:41:58.761861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11af79b-c953-4229-8f8e-e370d272b1b0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:41:58.866088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:41:58.866723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:41:58.867002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:41:58.867272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] PATH_INFO: `/v3/projects/c2debd4dad5348a591cca81bf1ad96e4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:41:58.877790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2debd4dad5348a591cca81bf1ad96e4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:41:58.878840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:41:58.878840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:41:58.878840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:41:58.880030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:41:59.010778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2debd4dad5348a591cca81bf1ad96e4', 'request_id': 'req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:59.024568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2debd4dad5348a591cca81bf1ad96e4', 'request_id': 'req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:59.030776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2debd4dad5348a591cca81bf1ad96e4', 'request_id': 'req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:59.114028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] The token cache is being invalidate because project c2debd4dad5348a591cca81bf1ad96e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:41:59.114347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:59.114774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f83f59a3-d10e-409c-a7f2-5b2a9f8cd17e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:41:59.116181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 767/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:41:58 2022] DELETE /identity/v3/projects/c2debd4dad5348a591cca81bf1ad96e4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:42:08.276752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=62 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:42:09.111946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:43:35.275784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:35.276345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:35.276345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:35.277602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:35.282568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9953cf-9084-4e2e-9b69-f4066e52e7f7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:35.283585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9953cf-9084-4e2e-9b69-f4066e52e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:35.283864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9953cf-9084-4e2e-9b69-f4066e52e7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:35.367553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8a9953cf-9084-4e2e-9b69-f4066e52e7f7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:35.368755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8a9953cf-9084-4e2e-9b69-f4066e52e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:35.472204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:35.629893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:35.632064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:35.750120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7bef-f59c-472e-80f4-5b91bf533570 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:35.756173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:35.756781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:35.757138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:35.757402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:35.765508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:35.767727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0522d6-97af-4abf-a259-b2fd9ff36318 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:35.777276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 745/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:43:36.873750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349045-7d55-4a59-9572-7dbfe47dd37c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:36.874235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349045-7d55-4a59-9572-7dbfe47dd37c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:36.874342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349045-7d55-4a59-9572-7dbfe47dd37c None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:36.875643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:36.881860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f91a-011e-464c-aec7-9a8cd621d47d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:36.882334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f91a-011e-464c-aec7-9a8cd621d47d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:36.882446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f91a-011e-464c-aec7-9a8cd621d47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:36.902453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3fb2f91a-011e-464c-aec7-9a8cd621d47d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:36.903654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3fb2f91a-011e-464c-aec7-9a8cd621d47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:36.912048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:37.067337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349045-7d55-4a59-9572-7dbfe47dd37c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:37.082433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349045-7d55-4a59-9572-7dbfe47dd37c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:37.089219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:37.089760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:37.090040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:37.090341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:37.099015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:37.100589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2fbd5-7d53-4980-9d16-e926bdfea47f service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:37.109840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 770/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:43:37.116703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faae19e-b669-42b6-a44b-89274425f685 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:37.116984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faae19e-b669-42b6-a44b-89274425f685 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:37.117253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faae19e-b669-42b6-a44b-89274425f685 None None] PATH_INFO: `/v3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:37.118542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 747/1517] 10.222.0.61 () {54 vars in 925 bytes} [Fri Sep 9 11:43:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:43:37.124523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d128-e848-486e-950b-8d2c0d6c231b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:37.124807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d128-e848-486e-950b-8d2c0d6c231b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:37.125080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d128-e848-486e-950b-8d2c0d6c231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:37.193358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6d09d128-e848-486e-950b-8d2c0d6c231b None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:37.194778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6d09d128-e848-486e-950b-8d2c0d6c231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:37.213322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:43:37.269642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 771/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Fri Sep 9 11:43:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:46.821443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c352bf-2020-4242-b6fa-b2262ad9e98e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:46.821892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c352bf-2020-4242-b6fa-b2262ad9e98e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:46.822001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c352bf-2020-4242-b6fa-b2262ad9e98e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:46.823353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 748/1519] 10.222.0.60 () {54 vars in 961 bytes} [Fri Sep 9 11:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:46.828627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2879-4764-426c-b073-ae283b6a6484 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:46.829053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2879-4764-426c-b073-ae283b6a6484 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:46.829170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf2879-4764-426c-b073-ae283b6a6484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:46.849192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ddcf2879-4764-426c-b073-ae283b6a6484 None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:46.850421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ddcf2879-4764-426c-b073-ae283b6a6484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:46.858933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 772/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Sep 9 11:43:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:43:46.867025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c352bf-2020-4242-b6fa-b2262ad9e98e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:46.880545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c352bf-2020-4242-b6fa-b2262ad9e98e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:46.885334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:46.885913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:46.886218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:46.886491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:46.895067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:46.896616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850e9b2-cd96-4072-9000-8cd287fc13b3 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:46.906251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 749/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:43:48.597900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be3638-5dd0-4d0a-ae21-c8be7bc5f898 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:48.598505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be3638-5dd0-4d0a-ae21-c8be7bc5f898 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:48.598505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be3638-5dd0-4d0a-ae21-c8be7bc5f898 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:48.599796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:48.605927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f390-a843-41b8-a9b2-fb943657bb8d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:48.606354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f390-a843-41b8-a9b2-fb943657bb8d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:48.606501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692f390-a843-41b8-a9b2-fb943657bb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:48.626612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e692f390-a843-41b8-a9b2-fb943657bb8d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:48.627821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e692f390-a843-41b8-a9b2-fb943657bb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:48.635926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:48.788359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be3638-5dd0-4d0a-ae21-c8be7bc5f898 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:48.803044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be3638-5dd0-4d0a-ae21-c8be7bc5f898 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:48.808521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:48.809108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:48.809385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:48.809656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:48.818705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:48.820266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af26382-e04a-49d0-a51c-c1b3a983bbb5 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:48.829601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 774/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:43:49.890822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b1415-2eb3-4231-82bf-f5e2e6228f62 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:49.891372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b1415-2eb3-4231-82bf-f5e2e6228f62 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:49.891372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b1415-2eb3-4231-82bf-f5e2e6228f62 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:49.892736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 11:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:49.898999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d09896-a7fe-4372-9645-6db775d9bbbe None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:49.899446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d09896-a7fe-4372-9645-6db775d9bbbe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:49.899560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d09896-a7fe-4372-9645-6db775d9bbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:49.919056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a8d09896-a7fe-4372-9645-6db775d9bbbe None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:49.920246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a8d09896-a7fe-4372-9645-6db775d9bbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:49.928665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 11:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:50.079895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b1415-2eb3-4231-82bf-f5e2e6228f62 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:50.094557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b1415-2eb3-4231-82bf-f5e2e6228f62 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:50.099973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:50.100560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:50.100838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:50.101110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:50.109094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:50.111345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32303e-e976-4eed-8e42-96b5d184bbd1 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:50.120618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 752/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:43:54.274149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:54.274756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:54.274756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:54.276007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:54.281174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633bd9-f443-4f06-9c9c-b270f31c61ed None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:54.281585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633bd9-f443-4f06-9c9c-b270f31c61ed None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:54.281716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7633bd9-f443-4f06-9c9c-b270f31c61ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:54.300770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e7633bd9-f443-4f06-9c9c-b270f31c61ed None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:54.302075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e7633bd9-f443-4f06-9c9c-b270f31c61ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:54.310100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:43:54.594275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:54.596018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:54.672632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baca9d6-8991-463d-9039-1844cb080775 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:54.676179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:54.676772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:54.677053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:54.677326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:54.686407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:54.688473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f1788-ab78-4b84-9949-72e751635e90 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:54.697751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 777/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:43:56.841934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d21ae4-7881-4973-8c47-88e33b447ffb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:56.842515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d21ae4-7881-4973-8c47-88e33b447ffb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:56.842515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d21ae4-7881-4973-8c47-88e33b447ffb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:56.844307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:43:56.849692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b837-5b57-408b-949a-2584cc775dc3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:56.850168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b837-5b57-408b-949a-2584cc775dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:56.850782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b837-5b57-408b-949a-2584cc775dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:56.870187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8e84b837-5b57-408b-949a-2584cc775dc3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:43:56.871367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8e84b837-5b57-408b-949a-2584cc775dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:43:56.879141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:43:57.165454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d21ae4-7881-4973-8c47-88e33b447ffb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:43:57.176152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d21ae4-7881-4973-8c47-88e33b447ffb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:43:57.179686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:43:57.180269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:43:57.180553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:43:57.180818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:43:57.189851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:43:57.191302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1c19c-b2b8-4baf-8bc2-d3516c442031 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:43:57.200634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 755/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:44:00.863227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:00.863702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:00.863818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:00.865109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:44:00.870429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032f2c-98f7-4c14-aa9d-7134e321a940 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:00.870853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032f2c-98f7-4c14-aa9d-7134e321a940 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:00.870963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56032f2c-98f7-4c14-aa9d-7134e321a940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:00.878438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=81 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:44:00.891594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-56032f2c-98f7-4c14-aa9d-7134e321a940 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:44:00.892777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-56032f2c-98f7-4c14-aa9d-7134e321a940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:44:00.900535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:44:01.203397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:44:01.205094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:44:01.271145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:44:01.284444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3bd18-1ffd-4b62-a4f9-2c8e351abe6b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:44:01.288424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:44:01.289056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:01.289333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:01.289714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:01.299086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:44:01.300551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5db2-beef-4443-bcfe-e52f52852706 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:44:01.310126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 780/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:44:03.448220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:03.448771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:03.448771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:03.450179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:44:03.455541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9a9b-a984-4411-855f-1eb304bba50d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:03.456018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9a9b-a984-4411-855f-1eb304bba50d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:03.456129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b9a9b-a984-4411-855f-1eb304bba50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:03.475381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-837b9a9b-a984-4411-855f-1eb304bba50d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:44:03.476567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-837b9a9b-a984-4411-855f-1eb304bba50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:44:03.485140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:44:03.790877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:44:03.792741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:44:03.806074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147f4c-f9a1-4f95-b1f6-ab4ad1e2c904 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:44:03.809525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:44:03.810137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:44:03.810672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:44:03.810941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:44:03.819993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:44:03.821486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1852-f418-49dc-8d9c-4c45eb8f3e4f service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:44:03.831437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 758/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:44:13.470180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=11 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:44:13.827323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=8 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:31.859962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c117b-c0ee-4e21-b140-806b52e48cb3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:31.862976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c117b-c0ee-4e21-b140-806b52e48cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:31.862976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c117b-c0ee-4e21-b140-806b52e48cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:31.888345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a60c117b-c0ee-4e21-b140-806b52e48cb3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:31.889825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a60c117b-c0ee-4e21-b140-806b52e48cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:31.903207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 782/1540] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:31.913403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:31.914010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:31.914010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:31.946103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:31.947342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:31.952498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66871a33-51c7-46b0-9220-8a8998a423fb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:31.952797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66871a33-51c7-46b0-9220-8a8998a423fb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:31.953060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66871a33-51c7-46b0-9220-8a8998a423fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:31.971212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 759/1541] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:31.976727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:31.984909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-66871a33-51c7-46b0-9220-8a8998a423fb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:31.986187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66871a33-51c7-46b0-9220-8a8998a423fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:31.992388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddbcaf-6cd3-42ef-9982-58647d6d0cdc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:31.999104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 783/1542] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:45:32.008750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.013151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.013274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.013396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.013482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.013482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.013482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.013482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.013482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.013905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:32.013905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:45:32.013905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:45:32.013905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aadabe-2544-4093-8d77-3f4da5227485 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.042312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 760/1543] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:45:32.047544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.047671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.047671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.047839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.047925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.061224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 784/1544] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:45:32.062048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.081210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addb096-903f-4d23-8ca0-44fdfd203bcc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.083267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.084485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.102359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 761/1545] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.102576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.102576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.102576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.102576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.103001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.103085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.103186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:32.103186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:45:32.103186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:45:32.103918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d7375-b213-45f6-97fa-24c7cbb1a596 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.120401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de109e-2457-4064-9adc-8b4c545a43eb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.130600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.131223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.131563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.131760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.132409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.133428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.139363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01d707-85a0-47ea-b0bd-0a90825b85bb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.157469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 762/1546] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.159883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.160175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.160453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.199163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.200471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.219834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 763/1547] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.220688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.238993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8e411-9260-4117-9090-4e7239bc713a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.245313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.247249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.247249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.247249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.247249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.248997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f6804-8ff3-4293-93a5-773555de4dc0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.273275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 785/1548] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/projects => generated 405 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:32.276055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.276055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.276055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.295114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.296325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.354936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 786/1549] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.355770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.364981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed73dbd-b173-4f7d-8146-b4dc91cb5bc8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.370224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.370743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 764/1550] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:32.370873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.371100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.371374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.371894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.372024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.372179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.372317 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.372445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.374086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71647ec3-95b7-46df-b36e-b5a07e7b2cf0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.386279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 787/1551] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.387310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bbfbf-61cc-4beb-8c65-56547c26238a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.387592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bbfbf-61cc-4beb-8c65-56547c26238a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.387863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bbfbf-61cc-4beb-8c65-56547c26238a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.405091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.406566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-484bbfbf-61cc-4beb-8c65-56547c26238a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.406677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.407722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-484bbfbf-61cc-4beb-8c65-56547c26238a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.468400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 788/1552] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.469556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 765/1553] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.469701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c68b-771c-4ab6-8f5e-2b9cfdc74be7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.477657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c68b-771c-4ab6-8f5e-2b9cfdc74be7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.477843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.477949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36c68b-771c-4ab6-8f5e-2b9cfdc74be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.488298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eed5e8-701a-4cd9-83ee-699eec73a7ca None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.493307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.493909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.494243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.494535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.495166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.495313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dc36c68b-771c-4ab6-8f5e-2b9cfdc74be7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.496253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:32.496253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:45:32.496253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:45:32.496514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dc36c68b-771c-4ab6-8f5e-2b9cfdc74be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.497452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.504387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 789/1554] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.506002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee80b1-d404-494c-b5f5-5e4c44ed712d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.506899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee80b1-d404-494c-b5f5-5e4c44ed712d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.507199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee80b1-d404-494c-b5f5-5e4c44ed712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.527562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d8ee80b1-d404-494c-b5f5-5e4c44ed712d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.528744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d8ee80b1-d404-494c-b5f5-5e4c44ed712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.536439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 790/1555] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.537574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.537856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.538154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.558602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.559782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.567564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 791/1556] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.568297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.578406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef566ee-5977-46c6-a1c0-2e3f7d31394a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.580889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 766/1557] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:32.581776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.583135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.583873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.583989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.584322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.584877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.586828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a245a-7f2e-4017-8668-94e79be39044 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.591854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42465c0-c1d9-4f99-b4af-dbcf4b317545 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.597658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.597658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.597658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.597658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.598186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.600076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec4f3f-0695-4202-b85e-5aea987dfa1f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.678804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:32.684680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 792/1558] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:32.687361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99147-6ab8-4164-bb07-80ac86509a99 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.687361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99147-6ab8-4164-bb07-80ac86509a99 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.687361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c99147-6ab8-4164-bb07-80ac86509a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.706682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f0c99147-6ab8-4164-bb07-80ac86509a99 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.707916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f0c99147-6ab8-4164-bb07-80ac86509a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.715021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 767/1559] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:32.715021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04091eb-654a-48d6-8eef-49674a34ef48 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.718079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04091eb-654a-48d6-8eef-49674a34ef48 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.718079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04091eb-654a-48d6-8eef-49674a34ef48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.718252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 793/1560] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.718548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.718840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.719115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.735985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a04091eb-654a-48d6-8eef-49674a34ef48 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.737189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a04091eb-654a-48d6-8eef-49674a34ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.738958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.756575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.784512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:32.797523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 768/1561] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.801919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 794/1562] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.816165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.816321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.816414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.816414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.816600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056da8-a9eb-4f80-ba81-4d876a74561e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.817808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.818451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.818739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.819033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.819686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.820019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.821614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.824701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:32.825943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:32.841952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 795/1563] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.841952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.848326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 769/1564] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:32.848326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.848524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040b77a-ddd5-4146-b922-9a93e6151264 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.848918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053ce83-ba5d-4b8a-b209-2f59e22e33ee None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.850695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.851313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.851595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.851868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.852534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.853488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.854623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.854736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.854901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.855172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.855817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.857441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.865191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 796/1565] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:32.866145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.867562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 770/1566] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:32.868449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.878798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7706-9ee0-4087-9616-90fa69b358a4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.879912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646a1c-8d17-4453-b170-a394acbbd26b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.883365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.883997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.889228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.889363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.889475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.889582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.889680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.889775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.889878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.889981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.889981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.890205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.890295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.901780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 771/1567] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.911729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.919966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ec54e-7ea3-436d-945d-eb4569ad1090 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.924540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.925114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.925421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.925705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.926483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.926787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.928367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.940646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 772/1568] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.941348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.951115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29f881-c0a6-4e8b-94b0-652bd61d5277 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.955720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.956286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.956563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.956835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.957456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.957758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.959389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:32.972115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 773/1569] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:32.974652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:32.982951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d60e7e-85bc-48e4-82e9-2bb210aef548 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:32.987552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:32.988111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:32.988679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:32.988956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:32.989570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:32.989870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:32.991451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.003158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 774/1570] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:33.003897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.013307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4e049-9e92-49cb-92e9-8de583049ced admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.018478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.019054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.019333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.019604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.020218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.023366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 797/1571] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:32 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.023366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.023589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.031721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75df68-e74b-4f46-9e35-79aeb242812f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.036532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.049349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.049349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.049349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.049349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.049349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.050589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 798/1572] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:33.051377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.063606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac71f2-f808-4e1b-a280-db5e80295f89 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.068128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.068689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.068966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.069239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.069860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.071449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.081545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 799/1573] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:33.083707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.091390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fcdb-64e3-48cd-b0be-fb836446c1d1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.095919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.096465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.096742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.097010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.097614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.099452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.143169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 775/1574] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.143169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.204427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9882d-55f6-4e3a-8e2b-7c29a83ea887 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.208949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.209535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.209809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.210098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.210715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.212654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.253321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 800/1575] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.253321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.314455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd28c2-a9ec-44ff-a6de-fbab13c00611 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.319179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.319769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.320043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.320310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.320933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.322972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.337247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:33.561612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 801/1576] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/projects => generated 389 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.562376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.622085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb35e0-22a6-4f49-b7bf-08e1a93b7d9e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.626657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.627251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.627530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.627804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.628431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.630353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.639224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 776/1577] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/projects => generated 377 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.647197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.689043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:33.703210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965997e-830d-4a53-aa35-5a5681f8b91f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.707893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.708489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.708771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.709044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.709685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.711314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.721388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 777/1578] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:33.724114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.731766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9003-c75b-49ea-a1fd-5ef907be41b8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.736389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.737036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.737314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.737587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] PATH_INFO: `/v3/projects/dbbb5c7f625c42e9b811cf39baad7d45/users/23c67dc88ee44238bf65272854c230c9/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.752866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 802/1579] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:33.752866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.755039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbb5c7f625c42e9b811cf39baad7d45, user_id=23c67dc88ee44238bf65272854c230c9, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.758222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.813883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acadd-89b3-4c0d-82bb-2b08085a3209 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.818671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.819324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.819604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.819881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] PATH_INFO: `/v3/projects/05fdf6e9eb54460aa442c83706b1238e/users/41c9e5d537ab4a4a8931311d2dff3f13/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.834996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05fdf6e9eb54460aa442c83706b1238e, user_id=41c9e5d537ab4a4a8931311d2dff3f13, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.836009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:33.836009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:45:33.836009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:45:33.837954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:33.903552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 778/1580] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:33 2022] PUT /identity/v3/projects/dbbb5c7f625c42e9b811cf39baad7d45/users/23c67dc88ee44238bf65272854c230c9/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:33.905096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.920186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 803/1581] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:33 2022] PUT /identity/v3/projects/05fdf6e9eb54460aa442c83706b1238e/users/41c9e5d537ab4a4a8931311d2dff3f13/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:33.921193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:33.970803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd68bd0-a3b5-4b31-bad9-67fcb45bcc65 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.983707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd85af-3cfd-49d5-8454-5d12bc969725 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:33.988277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:33.988874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:33.989152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:33.992256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:33.992256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:33.992256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.001360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:34.028629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.029234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.029511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.029779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.030438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.032385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.117060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 804/1582] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:34.117060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.125078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f44a-48ac-4faf-8351-c7738268c102 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.129820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.130444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.130822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.131093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.131826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.133833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.143817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 779/1583] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:33 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:34.143817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.153567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38678cd-aeea-4094-8ec7-0453c4adac1a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.158193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.158762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.159040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.159315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.159929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.161821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eab1ec-13f4-4c4c-bf1a-26f53d83f96f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.231839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 805/1584] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:34.231839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.239304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb36b4-127e-4c8c-8f38-79288d1eb105 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.243818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.244541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.244818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.245085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.245713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.247298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.258643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 806/1585] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.267396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.271003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfe491-ca73-43e0-b3b4-892682cb941b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.276205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.276795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.277075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.277349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.278122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.279994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.295580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 780/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:34.298593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7b8c-14f6-4a1b-8891-60d010172749 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.298593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7b8c-14f6-4a1b-8891-60d010172749 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.298593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7b8c-14f6-4a1b-8891-60d010172749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.316445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-04cf7b8c-14f6-4a1b-8891-60d010172749 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:34.317629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-04cf7b8c-14f6-4a1b-8891-60d010172749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:34.325862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 781/1587] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:34.327239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.327342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.327604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.346012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:34.347209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:34.355228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 782/1588] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:34.356015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.367805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f74c95-c9b7-47f1-8922-0add376371bb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.372352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.372948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.373226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.373496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.374174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.375775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.387374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 807/1589] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:34.387374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.390804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 783/1590] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.390804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.401471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a80648-63b7-4c5c-9b83-780bbc2120d4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.402158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a79e0c-c04c-4672-8812-94931f770257 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.406084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.406718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.406996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.407326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.407444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.407918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.408038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.408247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.408521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.409160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.409492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.410777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.419813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 808/1591] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.420835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.420978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 784/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.421489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.426332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:34.431622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae81af-d864-416a-bbbf-90515dbff31c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.431806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9e06-a47b-4270-bcc9-9ad4fb0215af admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.436154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.439636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.439773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.439871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.439962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.440052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.440157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] PATH_INFO: `/v3/projects/267962bfc8234bbeac2bebfa3e36d3fa/users/6f7d6dbe27ba42169a0f26d58c68f50c/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.440257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.440257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.440257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.450641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 785/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.451445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.454398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:34.456870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=267962bfc8234bbeac2bebfa3e36d3fa, user_id=6f7d6dbe27ba42169a0f26d58c68f50c, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.459555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.511446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6af88-19d5-4e7d-b84b-b88e55f442e4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.515097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.515799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.516086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.516359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.522658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 809/1594] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:34 2022] PUT /identity/v3/projects/267962bfc8234bbeac2bebfa3e36d3fa/users/6f7d6dbe27ba42169a0f26d58c68f50c/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:34.529679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.529903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.531404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.541561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 786/1595] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:34.542536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.551400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b744c82-0f59-4df3-95fd-6c8b64bce079 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.589983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a0881-7dde-41bf-bbdf-b8efa7c9cddc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.593804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.594447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.594732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.595002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.609868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.610503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.610827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.611099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.611760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.613447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.627138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 787/1596] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.627138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.636223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eb60e-7fc8-4bb9-8755-95aaa0619394 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.641408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.644439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.644439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.644439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.644439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.645017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.658897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 788/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.659920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.666551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.668325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.671078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e9be-b0bd-49cc-9c64-083aa5d8758d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.676708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.677306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.677646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.677924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.678599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.679251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 810/1598] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:34.680019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.680201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.691512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15516607-7bc9-4d70-ad2b-c1b9ff400c31 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.698692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 789/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.698692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.699837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.700450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.700727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.700994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.701630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.703424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc6d09-5482-4834-acaa-31416b7d83ea admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.710254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee03-f1fc-46b9-a11c-f6b1205ecfaa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.721755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.721755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.721755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.721755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.722164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.722241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 811/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.722334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.722421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.722421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.722421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.732754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 790/1601] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.732754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.743283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f2843-d873-4eef-aa53-53a671785d97 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.746848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:34.750349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.750349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.750773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.751064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] PATH_INFO: `/v3/projects/be8191545f754abbbb1cf75b94c1961a/users/d614a216aa0c42a983af191117871a08/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.763344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] MFA Rules not processed for user `6f7d6dbe27ba42169a0f26d58c68f50c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:34.764617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:34.770430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8191545f754abbbb1cf75b94c1961a, user_id=d614a216aa0c42a983af191117871a08, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.773215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.841255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 812/1602] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:34.841255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.854775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b9bb5-4164-4618-a317-4ed55a2dc87c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.859700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.860306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.860588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.861231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.862940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.864680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.870318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 791/1603] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:34 2022] PUT /identity/v3/projects/be8191545f754abbbb1cf75b94c1961a/users/d614a216aa0c42a983af191117871a08/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:34.871458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.880265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 813/1604] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:34.881356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:34.893469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878cea-db6b-4993-86bb-7f9c5fdd2d81 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:34.943641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:34.969180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:34.970282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:34.970589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:34.970937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] PATH_INFO: `/v3/projects/b15cb81206da4818a54b81f43187348f/users/259cacf455414fe8ab47f5c10b42871a/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:34.990680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15cb81206da4818a54b81f43187348f, user_id=259cacf455414fe8ab47f5c10b42871a, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:34.993498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:34.995694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac6035-34f8-4980-bf18-37dd02bc08b8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.003034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.003757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.004038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.004310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] PATH_INFO: `/v3/projects/0b33edf3f3594215a86fa6d2d0e68c77/users/e00238b83af948fe8752f998aa08b0b6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.023234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b33edf3f3594215a86fa6d2d0e68c77, user_id=e00238b83af948fe8752f998aa08b0b6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.026354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.154724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 792/1606] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:34 2022] PUT /identity/v3/projects/0b33edf3f3594215a86fa6d2d0e68c77/users/e00238b83af948fe8752f998aa08b0b6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:35.156189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.167601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 814/1606] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:34 2022] PUT /identity/v3/projects/b15cb81206da4818a54b81f43187348f/users/259cacf455414fe8ab47f5c10b42871a/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:35.168640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.236368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd04167-7d8e-438d-a698-6145b509fb01 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.242880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.246246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.246246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.246246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] PATH_INFO: `/v3/projects/9ac7346b9af144f48f0760796c594016/users/683fe1045ddb4bc1a8a94d3948495f96/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.258984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa5b48-45b4-4e34-a9a1-959e747fafe3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.271400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac7346b9af144f48f0760796c594016, user_id=683fe1045ddb4bc1a8a94d3948495f96, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.274918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44135e-d428-4ae8-9cff-a3d505380e75 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.288997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:35.316091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.316727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.317007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.317278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.368373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 793/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:35 2022] PUT /identity/v3/projects/9ac7346b9af144f48f0760796c594016/users/683fe1045ddb4bc1a8a94d3948495f96/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:35.371162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935396a-ff56-450c-8198-7a48d40a1f27 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.371515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935396a-ff56-450c-8198-7a48d40a1f27 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.371884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935396a-ff56-450c-8198-7a48d40a1f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.383287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.386011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.397117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 815/1608] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:35.398957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.401035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8935396a-ff56-450c-8198-7a48d40a1f27 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:35.403329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8935396a-ff56-450c-8198-7a48d40a1f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:35.415232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d711d6-d0fa-42fb-9018-8b4c019494a7 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.483538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.485313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.486472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.487304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] PATH_INFO: `/v3/projects/c7dd050f40b8417cb0a00f672d629f80/users/3a61e50604ad4c8baebe7fe29719702d/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.502348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 794/1609] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:35.503743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f900f-8d08-4ddf-9c22-636e22bdd487 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.504032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f900f-8d08-4ddf-9c22-636e22bdd487 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.504311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f900f-8d08-4ddf-9c22-636e22bdd487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.510180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7dd050f40b8417cb0a00f672d629f80, user_id=3a61e50604ad4c8baebe7fe29719702d, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.515520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933347b3-5bc1-4b80-9712-0b4967497db8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.525823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c20f900f-8d08-4ddf-9c22-636e22bdd487 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:35.527176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c20f900f-8d08-4ddf-9c22-636e22bdd487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:35.535491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 795/1610] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:35.536712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.536998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.537271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.569679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] MFA Rules not processed for user `259cacf455414fe8ab47f5c10b42871a`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:35.571409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:35.601674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:35.619185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 816/1611] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:35 2022] PUT /identity/v3/projects/c7dd050f40b8417cb0a00f672d629f80/users/3a61e50604ad4c8baebe7fe29719702d/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:35.621341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.621890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.622421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.653063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:35.654469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:35.672393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 796/1612] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:35.675074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.759292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d609d68-cd3f-4e70-8139-ffaca6bc57e4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.759767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 817/1613] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:35.760653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.771753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:35.822895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.823515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.823831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.824104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.830086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaad79-bf63-4784-8249-3f582682a10b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.834437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.836473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a48c7-48a0-4d0f-b78c-aca4984a075b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.848188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 797/1614] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:35.849288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff36e-9322-46f3-a570-f83c5bb3650f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.849657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff36e-9322-46f3-a570-f83c5bb3650f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.849937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ff36e-9322-46f3-a570-f83c5bb3650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.851978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.852777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.853202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.855230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.869883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.871659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4746ead-e1ad-440a-ba0c-0bab3da0a626 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.885337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 818/1615] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:35.887046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54063-b1f0-4bbe-98d9-a03252e9a924 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.887338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54063-b1f0-4bbe-98d9-a03252e9a924 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.887615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f54063-b1f0-4bbe-98d9-a03252e9a924 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.889135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 819/1616] 10.222.0.60 () {54 vars in 962 bytes} [Fri Sep 9 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:35.890075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f54063-b1f0-4bbe-98d9-a03252e9a924 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:35.894079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-886ff36e-9322-46f3-a570-f83c5bb3650f None None] MFA Rules not processed for user `3a61e50604ad4c8baebe7fe29719702d`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:35.895619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-886ff36e-9322-46f3-a570-f83c5bb3650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:35.906525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f54063-b1f0-4bbe-98d9-a03252e9a924 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:35.911500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:35.912643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.913095 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.913517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.924946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:35.927149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d34f43-c742-433d-9824-a21c83e97489 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:35.939995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 820/1617] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:35.941437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:35.941567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:35.941886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:35.995572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 798/1618] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:36.021005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:36.027931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:36.035535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6169cb-c6a0-4c13-858b-da4054b0783b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.036049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6169cb-c6a0-4c13-858b-da4054b0783b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.036049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6169cb-c6a0-4c13-858b-da4054b0783b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.037518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 799/1619] 10.222.0.60 () {54 vars in 962 bytes} [Fri Sep 9 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:36.047374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb31a-2231-49a5-88a1-fe57ccb079dc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.047645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb31a-2231-49a5-88a1-fe57ccb079dc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.051030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb31a-2231-49a5-88a1-fe57ccb079dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.075617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-943fb31a-2231-49a5-88a1-fe57ccb079dc None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:36.076920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-943fb31a-2231-49a5-88a1-fe57ccb079dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:36.162831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 821/1620] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Sep 9 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:36.174898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:36.188584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0a41e-f653-45cc-9575-e36be272571d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:36.203901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:36.205045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.205342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.205596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.234646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 800/1621] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:36.236753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.237044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.237336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.325759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] MFA Rules not processed for user `23c67dc88ee44238bf65272854c230c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:36.327441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:36.333604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:36.335270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:36.346402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 822/1622] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:45:36.347270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:36.364006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8afb-57a4-47c8-b6af-ebc75a523b3e service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:36.371073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:36.371846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.372191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.372638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.389346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:36.391420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:36.405735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 823/1623] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:36.473815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:36.506350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 801/1624] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:36.510513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:36.521717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788936-c456-4745-a752-bb3eb58365d8 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:36.527129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:36.528426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.528705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.529096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.530594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:36.533727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce7cb9-aba3-40a6-84c1-b778c407b0ce admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:36.669451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:36.675708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 824/1625] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/projects => generated 403 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:36.685083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3765d-cdc8-476b-a10c-3620e14eaea0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:36.806279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937923-c4e1-497c-bb70-21e51ce5d3a6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.807190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937923-c4e1-497c-bb70-21e51ce5d3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.807465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937923-c4e1-497c-bb70-21e51ce5d3a6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.808925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 825/1626] 10.222.0.60 () {54 vars in 912 bytes} [Fri Sep 9 11:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:45:36.816384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae576-04d0-445d-b321-4eda5c519f55 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.817028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae576-04d0-445d-b321-4eda5c519f55 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.817028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ae576-04d0-445d-b321-4eda5c519f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.838894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:36.839531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.839812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.840079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.840708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:36.846021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7c1ae576-04d0-445d-b321-4eda5c519f55 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:36.846586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:36.848209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7c1ae576-04d0-445d-b321-4eda5c519f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:36.919618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:36.946630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 826/1627] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:45:36.947899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027952c-27f8-449a-a71a-bf9fe3c8140a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.948184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027952c-27f8-449a-a71a-bf9fe3c8140a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.948459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027952c-27f8-449a-a71a-bf9fe3c8140a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.950088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 827/1628] 10.222.0.60 () {54 vars in 912 bytes} [Fri Sep 9 11:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:36.956084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:36.959222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:36.959222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:36.980146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:36.981504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:36.988431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 802/1629] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/users => generated 308 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:36.992430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.007062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 828/1630] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:37.008289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.019357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950248c-02f2-4393-82ac-fad37648f6c3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.024263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.024994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.025931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.026233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.027001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.028644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.041901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 829/1631] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:37.043511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.092490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb8754-e079-4c17-a584-8e777ab41695 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.097430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.098283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.098997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.100104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.115715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3cdac-1a06-4e05-8339-68e0c16eb215 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.116027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.117672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.121605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.122631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.122974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.123537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.386528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 803/1632] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:37.386528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.419961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.422353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812a84d-22e0-4eeb-b4ae-c421ebcfbc8d service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.422607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.423174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.423174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.423174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.423174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] PATH_INFO: `/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2/users/e0cf970a482940158d98a3521eebaa4d/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.423858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaf3e3d09e84c7fa6e0d8a27d9d26f2, user_id=e0cf970a482940158d98a3521eebaa4d, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.423858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.424296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 830/1633] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:37.424494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:37.424494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 804/1634] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:37 2022] PUT /identity/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2/users/e0cf970a482940158d98a3521eebaa4d/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:37.424793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.424793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015ec2-3cc4-412b-9f5a-85ba8ded39ee service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.480898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.481945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.482378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.482882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.483842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.485703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.495824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 831/1635] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:37.507287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:37.517991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a1162-bb10-4550-bcb9-09790a6f056d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:37.528074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:37.528751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:37.529073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:37.529856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] PATH_INFO: `/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2/users/e0cf970a482940158d98a3521eebaa4d/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:37.532573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaf3e3d09e84c7fa6e0d8a27d9d26f2, user_id=e0cf970a482940158d98a3521eebaa4d, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:37.535590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8c7da-3d66-4dd5-a77f-1e76cbc5c5e6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:37.546624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 805/1636] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:37 2022] PUT /identity/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2/users/e0cf970a482940158d98a3521eebaa4d/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:45:37.751578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:41.843287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:41.855779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc6f8c-8e32-41ce-8ea7-7f64beab5428 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:42.029928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:42.030637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:42.030969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:42.031253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] PATH_INFO: `/v3/users/259cacf455414fe8ab47f5c10b42871a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:42.033451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=259cacf455414fe8ab47f5c10b42871a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:42.044621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:42.044621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 09 11:45:42.044621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: Sep 09 11:45:42.044621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:42.406575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259cacf455414fe8ab47f5c10b42871a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:42.452067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259cacf455414fe8ab47f5c10b42871a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:42.478904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259cacf455414fe8ab47f5c10b42871a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:42.479955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259cacf455414fe8ab47f5c10b42871a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:42.575281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4e854316-e05e-4a47-93ec-9d00332d9fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259cacf455414fe8ab47f5c10b42871a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:42.587921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 832/1637] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:41 2022] DELETE /identity/v3/users/259cacf455414fe8ab47f5c10b42871a => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:42.589159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:42.589649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:42.589720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:42.687325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:42.688822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:42.815376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:42.815875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:42.815983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:42.854896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 806/1638] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:42.872551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:42.874191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:42.909517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] MFA Rules not processed for user `e0cf970a482940158d98a3521eebaa4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:42.928465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:42.986273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:43.002941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 833/1639] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:43.008176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e92230-c0c0-4ebf-bb56-9f6a4da58a82 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:43.021312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:43.022330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:43.022819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:43.023348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:43.036252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:43.038635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f295cc7-b0b4-4362-a3ac-4c1e5813f5c8 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:43.058921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 807/1640] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:44.208789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:44.211869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:44.251523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b8f9d-9c25-4e77-8ba0-3b09d2e32b00 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:44.263695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:44.264362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:44.264691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:44.264961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] PATH_INFO: `/v3/projects/b15cb81206da4818a54b81f43187348f` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:44.266171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15cb81206da4818a54b81f43187348f)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:44.268245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:44.473331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15cb81206da4818a54b81f43187348f', 'request_id': 'req-363e7b26-643a-4d96-8274-655ceb36a200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:44.489243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15cb81206da4818a54b81f43187348f', 'request_id': 'req-363e7b26-643a-4d96-8274-655ceb36a200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:44.497670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15cb81206da4818a54b81f43187348f', 'request_id': 'req-363e7b26-643a-4d96-8274-655ceb36a200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:44.601386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] The token cache is being invalidate because project b15cb81206da4818a54b81f43187348f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:44.601682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:44.603916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-363e7b26-643a-4d96-8274-655ceb36a200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:44.607491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 834/1641] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:44 2022] DELETE /identity/v3/projects/b15cb81206da4818a54b81f43187348f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:44.612109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:44.612593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:44.612678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:44.655337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:44.662021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:44.748296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:44.867239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 808/1642] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:44.877191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:44.877484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:44.877761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:44.921487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:44.924327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:44.949860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 835/1643] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:44.953863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.000874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c23c9-722a-41e7-9ecc-2426c0f3fef3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.015560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.023082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.024052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.024635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.042384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:45.043506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.061062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.094392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 809/1644] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:45.098432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.116990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2b90-bf06-49e3-8432-fada659edf4a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.134796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.135650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.135934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.136210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.136865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.138987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9c23c-16fb-4c8c-b60d-0f99937d9c19 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.297369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 836/1645] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:45 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:45.302092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.316195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244274dd-f661-41e9-a2d2-d66e953df72c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.392650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87fa1-8eef-4ac4-a772-74956d80703c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.393639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87fa1-8eef-4ac4-a772-74956d80703c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.394162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f87fa1-8eef-4ac4-a772-74956d80703c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.456568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-06f87fa1-8eef-4ac4-a772-74956d80703c None None] MFA Rules not processed for user `41c9e5d537ab4a4a8931311d2dff3f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:45.458898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-06f87fa1-8eef-4ac4-a772-74956d80703c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:45.462935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.463670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.464049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.464429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.468988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.468988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f886b-94ae-4cb1-8dd6-39cece39f4f3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.584107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 837/1646] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:45.587518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.588814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.589211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.629397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 810/1647] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:45 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:45.629397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b30feb-f8db-4f30-ba31-5fc810a1fbc3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.629397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b30feb-f8db-4f30-ba31-5fc810a1fbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.629397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b30feb-f8db-4f30-ba31-5fc810a1fbc3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.644932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 811/1648] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:45.644932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b30feb-f8db-4f30-ba31-5fc810a1fbc3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.645690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] MFA Rules not processed for user `41c9e5d537ab4a4a8931311d2dff3f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:45.645690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:45.646769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 838/1649] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:45.659369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b30feb-f8db-4f30-ba31-5fc810a1fbc3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.662622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.668071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:45.672247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.673410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.673933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.674320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.675724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.678322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.692186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 812/1650] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:45.696834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.715318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:45.716438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878cb3-b838-4fd4-91b0-300d4eabd4f1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.723462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.724001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.724288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.724564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.725221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.731697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74fb80-a2d6-4406-83eb-0212e08703f6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.754400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 813/1651] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:45:45.759689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.761568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.763544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.783478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675da04-6a6e-4476-b883-d96b78f7d8ce None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.792270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.793033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.793462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.794081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.795217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:45.798791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:45.834964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.837596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:45.872096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 839/1652] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:45.872942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.901128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 814/1653] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:45.934552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cbda3-b088-4427-ac01-b3ae40b41f16 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.942410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:45.944092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:45.948398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.949377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.949799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.950291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] PATH_INFO: `/v3/projects/a9a2b4aab409448795025c41310f1a9a/users/64c6ddbced314b479e63c272c72a603c/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.969382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e00f8-7321-4058-9945-41629665da8e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:45.984080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:45.984719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:45.985491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:45.987315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:45.993736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a2b4aab409448795025c41310f1a9a, user_id=64c6ddbced314b479e63c272c72a603c, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:45.996326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.007005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.009460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a32c16-b371-40a0-a309-f40b0deb6d88 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.039388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 815/1654] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:46.093143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 840/1655] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:45 2022] PUT /identity/v3/projects/a9a2b4aab409448795025c41310f1a9a/users/64c6ddbced314b479e63c272c72a603c/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:46.105395 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.105743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.105960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.127535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:46.137514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] MFA Rules not processed for user `64c6ddbced314b479e63c272c72a603c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:46.138848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:46.223014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 816/1656] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:46.235778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:46.326976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2acf-dcdc-4a41-abdd-89d9a7571515 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:46.334535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:46.335334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.337305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.337305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.337476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.339369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104466f-f10a-4add-9da2-5d1155951170 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.481846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 841/1657] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:46 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:46.486534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:46.505408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae95798-ebc5-4815-96ee-3d0ec24cc023 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.506437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae95798-ebc5-4815-96ee-3d0ec24cc023 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.506717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae95798-ebc5-4815-96ee-3d0ec24cc023 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.508155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 842/1658] 10.222.0.60 () {54 vars in 912 bytes} [Fri Sep 9 11:45:46 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:46.516736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68594d1-4d86-499a-8b3b-f7d3cdd92fb8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:46.521492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.522082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.523865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.621542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:46.622843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.623522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.623995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.625682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.634502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.644548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:46.648397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:46.734290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 817/1659] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:46 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:46.735365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:46.768543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47833a-d671-4b65-947b-9fc588b7bf5f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:46.786584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:46.787233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.787516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.791517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.791517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.791517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.795952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 843/1660] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:46.834634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 818/1661] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:46.839568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:46.856848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60a6d3-21bc-4cb1-97ee-b3fb4e61fe79 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:46.862959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:46.863798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.864079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.864425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] PATH_INFO: `/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5/users/d35ab06eec1b419797a9a493854f2a6a/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.896080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8632f4e8c9ba4d7dadf62ff5c41d43f5, user_id=d35ab06eec1b419797a9a493854f2a6a, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.897447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.913933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:46.916755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:46.935822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9227-a562-487b-82d7-e326623f7032 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:46.940019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:46.940878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:46.941985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:46.942828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:46.955821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:46.957696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:46.974523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 819/1662] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:47.095287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 844/1663] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:46 2022] PUT /identity/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5/users/d35ab06eec1b419797a9a493854f2a6a/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:47.099242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:47.117382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d4af-cb68-4922-929c-bd86057e890b service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:47.205563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:47.206612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:47.206955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:47.207230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:47.208397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:47.210847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e998-be72-4f45-94ec-c941c4e1af42 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:47.224405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 820/1664] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:47.229771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:47.243093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f578b79-a127-424f-b034-84c21800141d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:47.248986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:47.250723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:47.251130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:47.251473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] PATH_INFO: `/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5/users/d35ab06eec1b419797a9a493854f2a6a/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:47.253756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8632f4e8c9ba4d7dadf62ff5c41d43f5, user_id=d35ab06eec1b419797a9a493854f2a6a, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:47.256704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:47.275121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 845/1665] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:47 2022] PUT /identity/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5/users/d35ab06eec1b419797a9a493854f2a6a/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:47.283809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306e50-d17d-448b-b58b-5ba646b1eb81 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:47.284300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306e50-d17d-448b-b58b-5ba646b1eb81 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:47.284743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab306e50-d17d-448b-b58b-5ba646b1eb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:47.325605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ab306e50-d17d-448b-b58b-5ba646b1eb81 None None] MFA Rules not processed for user `d35ab06eec1b419797a9a493854f2a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:47.327305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ab306e50-d17d-448b-b58b-5ba646b1eb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:47.331335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:47.359440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:47.436832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 821/1666] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:47.443491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:47.446845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:47.591427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df256472-cdbf-4866-bf75-910b5abec832 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:47.600973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:47.601657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:47.604909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:47.605297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:47.631131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:47.638312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea6e7f-656e-4ef9-846a-9afbbe735e5c service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:47.660172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 846/1667] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:48.033629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b719-dc1d-4837-8de7-57488a02749d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.034637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b719-dc1d-4837-8de7-57488a02749d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.035333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53b719-dc1d-4837-8de7-57488a02749d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.039578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 822/1668] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:45:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:48.073762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061908aa-94ad-4cbf-aa42-52cc576d05d3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.074165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061908aa-94ad-4cbf-aa42-52cc576d05d3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.074349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061908aa-94ad-4cbf-aa42-52cc576d05d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.105580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-061908aa-94ad-4cbf-aa42-52cc576d05d3 None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:48.107195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-061908aa-94ad-4cbf-aa42-52cc576d05d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:48.278709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 847/1669] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:45:48.435088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b719-dc1d-4837-8de7-57488a02749d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:48.456506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53b719-dc1d-4837-8de7-57488a02749d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:48.463880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:48.464823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.466082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.466852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.468981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:48.502253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:48.503964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed5dba-bc32-4a7e-9999-153cf598cef2 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:48.515914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 823/1670] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:48.577194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf75e3-7ceb-4819-bcd5-a99b05a2dd69 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.577194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf75e3-7ceb-4819-bcd5-a99b05a2dd69 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.577194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf75e3-7ceb-4819-bcd5-a99b05a2dd69 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.577194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 848/1671] 10.222.0.60 () {54 vars in 961 bytes} [Fri Sep 9 11:45:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:45:48.581414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.581753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.582804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.646805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] MFA Rules not processed for user `fa0c95e5281f4f9d9dac910415d10932`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:48.649663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:48.673964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.674375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.674786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.685890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 824/1672] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Sep 9 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:48.692153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:48.723942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] MFA Rules not processed for user `41c9e5d537ab4a4a8931311d2dff3f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:48.731203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:48.747645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2029b-120f-4f30-a61f-2cdf49223150 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:48.778509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:48.779148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:48.779430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:48.779707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:48.809676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:48.811410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:48.878856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 849/1673] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:48.880672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:48.884391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:48.886708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 825/1674] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:48.939682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:49.069918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0ff1-9a6b-4ccd-aced-275226562b42 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:49.082154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:49.083411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:49.083766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:49.084663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] PATH_INFO: `/v3/users/23c67dc88ee44238bf65272854c230c9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:49.088243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c67dc88ee44238bf65272854c230c9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:49.093006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:49.317286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:49.330527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cba9-f196-4f6c-96bb-5122e5d7e0fd service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:49.338239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:49.339299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:49.339861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:49.340325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] PATH_INFO: `/v3/users/64c6ddbced314b479e63c272c72a603c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:49.342077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c6ddbced314b479e63c272c72a603c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:49.345508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:49.404717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c67dc88ee44238bf65272854c230c9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.455145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c67dc88ee44238bf65272854c230c9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.466035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c67dc88ee44238bf65272854c230c9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.467021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c67dc88ee44238bf65272854c230c9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.609709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-189c3e95-654d-4107-b15c-35fc19e9dc36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c67dc88ee44238bf65272854c230c9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.632877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 850/1675] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:48 2022] DELETE /identity/v3/users/23c67dc88ee44238bf65272854c230c9 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:49.660924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c6ddbced314b479e63c272c72a603c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.696036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c6ddbced314b479e63c272c72a603c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.716763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c6ddbced314b479e63c272c72a603c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.850378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c6ddbced314b479e63c272c72a603c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.851199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c6ddbced314b479e63c272c72a603c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:49.870792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:49.871298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:49.871743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:49.879539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 826/1676] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:49 2022] DELETE /identity/v3/users/64c6ddbced314b479e63c272c72a603c => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:49.937262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:49.943984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:50.099440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 851/1677] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:45:50.121486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:50.123041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:50.194587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:50.256919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa542d4-95cf-4558-a97f-1adce9436481 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:50.268691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:50.273581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:50.275022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:50.275721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:50.310730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:50.320426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:50.352988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 827/1678] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:50.546263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:50.547801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:50.572459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54153c2-9672-4e6b-a9ec-5e9fdf77ed50 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:50.581869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:50.582562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:50.582844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:50.588001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] PATH_INFO: `/v3/projects/dbbb5c7f625c42e9b811cf39baad7d45` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:50.590161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbbb5c7f625c42e9b811cf39baad7d45)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:50.592363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:50.877422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:50.932817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbbb5c7f625c42e9b811cf39baad7d45', 'request_id': 'req-0f49701a-5a86-4924-8e2d-367d0e06a3c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:50.964034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbbb5c7f625c42e9b811cf39baad7d45', 'request_id': 'req-0f49701a-5a86-4924-8e2d-367d0e06a3c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:50.992511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbbb5c7f625c42e9b811cf39baad7d45', 'request_id': 'req-0f49701a-5a86-4924-8e2d-367d0e06a3c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.138711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] The token cache is being invalidate because project dbbb5c7f625c42e9b811cf39baad7d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:51.139841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.155320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.165492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 852/1679] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:50 2022] DELETE /identity/v3/projects/dbbb5c7f625c42e9b811cf39baad7d45 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:51.169445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:51.190881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34a060-a9fd-422a-8f34-2491fa311c7c service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:51.428242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:51.430860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:51.433667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:51.434046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] PATH_INFO: `/v3/users/e0cf970a482940158d98a3521eebaa4d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:51.448415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0cf970a482940158d98a3521eebaa4d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:51.450521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:51.694953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:51.696954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:51.728148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0cf970a482940158d98a3521eebaa4d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.743293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49701a-5a86-4924-8e2d-367d0e06a3c8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:51.756497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:51.757158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:51.757442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:51.757722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] PATH_INFO: `/v3/projects/a9a2b4aab409448795025c41310f1a9a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:51.765211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a2b4aab409448795025c41310f1a9a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:51.770972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:51.785054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0cf970a482940158d98a3521eebaa4d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.810701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0cf970a482940158d98a3521eebaa4d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.895865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0cf970a482940158d98a3521eebaa4d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.897772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0cf970a482940158d98a3521eebaa4d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:51.910209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 828/1680] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:51 2022] DELETE /identity/v3/users/e0cf970a482940158d98a3521eebaa4d => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:51.995868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a2b4aab409448795025c41310f1a9a', 'request_id': 'req-a67cf749-232d-4873-9534-1ebf408f7bef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.020136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a2b4aab409448795025c41310f1a9a', 'request_id': 'req-a67cf749-232d-4873-9534-1ebf408f7bef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.044996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a2b4aab409448795025c41310f1a9a', 'request_id': 'req-a67cf749-232d-4873-9534-1ebf408f7bef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.159516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] The token cache is being invalidate because project a9a2b4aab409448795025c41310f1a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:52.159904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.160785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.163200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 853/1681] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:51 2022] DELETE /identity/v3/projects/a9a2b4aab409448795025c41310f1a9a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:45:52.167500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:52.169256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:52.335788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c699965-3df7-4bcd-9937-07e966bc654c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:52.485814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:52.486718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:52.487003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:52.487269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] PATH_INFO: `/v3/users/d35ab06eec1b419797a9a493854f2a6a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:52.490871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35ab06eec1b419797a9a493854f2a6a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:52.496167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:52.577894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:52.579816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:52.618335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf749-232d-4873-9534-1ebf408f7bef admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:52.632776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:52.633877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:52.634196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:52.634477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] PATH_INFO: `/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:52.644790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaf3e3d09e84c7fa6e0d8a27d9d26f2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:52.653630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:52.858605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaf3e3d09e84c7fa6e0d8a27d9d26f2', 'request_id': 'req-21213e96-7efe-44fb-bc7b-0d5640d557cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.860022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35ab06eec1b419797a9a493854f2a6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.877306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaf3e3d09e84c7fa6e0d8a27d9d26f2', 'request_id': 'req-21213e96-7efe-44fb-bc7b-0d5640d557cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.886817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaf3e3d09e84c7fa6e0d8a27d9d26f2', 'request_id': 'req-21213e96-7efe-44fb-bc7b-0d5640d557cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.907715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35ab06eec1b419797a9a493854f2a6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:52.923354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35ab06eec1b419797a9a493854f2a6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.044846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] The token cache is being invalidate because project cbaf3e3d09e84c7fa6e0d8a27d9d26f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:53.045324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.046529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.053162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 854/1682] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:52 2022] DELETE /identity/v3/projects/cbaf3e3d09e84c7fa6e0d8a27d9d26f2 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:45:53.053162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:53.055196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:53.173652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213e96-7efe-44fb-bc7b-0d5640d557cc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:53.180210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:53.181112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:53.181395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:53.181671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] PATH_INFO: `/v3/users/3a61e50604ad4c8baebe7fe29719702d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:53.184185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a61e50604ad4c8baebe7fe29719702d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:53.188005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:53.207563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35ab06eec1b419797a9a493854f2a6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.208328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d8e30d3-722e-4932-aa0a-95b767848715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35ab06eec1b419797a9a493854f2a6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.227705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 829/1683] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:52 2022] DELETE /identity/v3/users/d35ab06eec1b419797a9a493854f2a6a => generated 0 bytes in 1057 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:53.227705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68001eaa-7959-4889-8588-84cbe55d5ec9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:53.227705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68001eaa-7959-4889-8588-84cbe55d5ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:53.227705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68001eaa-7959-4889-8588-84cbe55d5ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:53.389234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-68001eaa-7959-4889-8588-84cbe55d5ec9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:53.399731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-68001eaa-7959-4889-8588-84cbe55d5ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:53.491834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a61e50604ad4c8baebe7fe29719702d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.543717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a61e50604ad4c8baebe7fe29719702d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.576021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a61e50604ad4c8baebe7fe29719702d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.577078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a61e50604ad4c8baebe7fe29719702d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.695988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e28371d2-97cc-4c8d-8405-09907775d646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a61e50604ad4c8baebe7fe29719702d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:53.725768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 855/1684] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:53 2022] DELETE /identity/v3/users/3a61e50604ad4c8baebe7fe29719702d => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:53.726588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 830/1685] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:53.737815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7c43d-f9ee-4240-8d88-a65355dff601 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:53.739073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7c43d-f9ee-4240-8d88-a65355dff601 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:53.756903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7c43d-f9ee-4240-8d88-a65355dff601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:53.761363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f86b-67cd-4d91-b263-8e693e205087 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:53.761687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f86b-67cd-4d91-b263-8e693e205087 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:53.761948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229f86b-67cd-4d91-b263-8e693e205087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:53.860786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d229f86b-67cd-4d91-b263-8e693e205087 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:53.862583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d229f86b-67cd-4d91-b263-8e693e205087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:53.866210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-66a7c43d-f9ee-4240-8d88-a65355dff601 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:53.870585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66a7c43d-f9ee-4240-8d88-a65355dff601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:53.995053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:54.037002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 831/1686] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:54.039068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.042246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.042246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.138313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 856/1687] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:54.141229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] MFA Rules not processed for user `683fe1045ddb4bc1a8a94d3948495f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:54.142755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.143044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.143323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.151485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:54.232102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] MFA Rules not processed for user `e00238b83af948fe8752f998aa08b0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:54.239930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:54.386699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 832/1688] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:54.387712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:54.417495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26871dd-20e6-4d30-9531-0e6a1a864725 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:54.434623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 857/1689] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:54.434623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:54.434623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:54.441814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:54.442709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.443372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.443372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.450421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:54.450744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:54.456250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:54.496526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 833/1690] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:54.498699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:54.508463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:54.570944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fe20e-7201-4061-ad05-d24a32311e2b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:54.581544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:54.590160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.590839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.592060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.598703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=19,keystone:SELECT=78,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:54.619167 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:54.622750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:54.637179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 858/1691] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:54.638240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:54.640053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:54.769816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88886686-cbb5-4602-855c-1e6b80c08976 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:54.787256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3288c2-5f8b-441a-bfce-e6459b1278a5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:54.788935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:54.790713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.791026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.794298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.798531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:54.798970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.799230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.799537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.806113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:54.807777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:54.827081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:54.831637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:54.838540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 859/1692] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:54.842608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:54.857371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 834/1693] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:54.861926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8a590-9869-4649-be76-956634a37fcf service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:54.884905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:54.890307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:54.890307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:54.890307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:54.890307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:54.891379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:55.089798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 860/1694] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:54 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:55.091060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:55.104726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86366ab3-74b9-4abc-a893-02a630f7c4d1 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:55.235616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:55.236303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.238071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.238397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.239061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:55.243845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:55.386524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 835/1695] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:55 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:45:55.395627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:55.410542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101091ef-1b47-4a10-a43f-9747c8a8fbda admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:55.442550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:55.442550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.442550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.442550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.443308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:55.445893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:55.452050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:55.458455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:55.483390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 861/1696] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:55.487695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:55.521945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816652e-1e8a-42d6-b3ef-73e3596a57fa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:55.527611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dd555-cb78-4af8-a7e0-ee4b6afdb735 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:55.542080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:55.542815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:55.543340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.544057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.545372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.546625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:55.548566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.549124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.549462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] PATH_INFO: `/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.551413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8632f4e8c9ba4d7dadf62ff5c41d43f5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:55.552661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 11:45:55.552661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 09 11:45:55.552661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: Sep 09 11:45:55.554328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:55.556502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f234ee-ae72-4a44-987b-835a75dbaaa6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:55.576039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 862/1697] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:55.578224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.580039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.580584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.684406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:55.692511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:55.813750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 863/1698] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:45:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:55.814753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:55.814858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8632f4e8c9ba4d7dadf62ff5c41d43f5', 'request_id': 'req-43810bb3-0224-4656-a0dd-93da9beba04c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:55.830153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e3f74-5972-473f-a026-79bfa53b5dcb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:55.930650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:55.931640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:55.932020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:55.934279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] PATH_INFO: `/v3/projects/917d00134c234e7695460ce39ad9360a/users/3d3a9ebe90a34b389d16eb3e01ca045c/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:55.973924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917d00134c234e7695460ce39ad9360a, user_id=3d3a9ebe90a34b389d16eb3e01ca045c, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:55.977324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d38896-c6a0-4a08-8082-9b4982313c77 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:56.010498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8632f4e8c9ba4d7dadf62ff5c41d43f5', 'request_id': 'req-43810bb3-0224-4656-a0dd-93da9beba04c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:56.020343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8632f4e8c9ba4d7dadf62ff5c41d43f5', 'request_id': 'req-43810bb3-0224-4656-a0dd-93da9beba04c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:56.064530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] The token cache is being invalidate because project 8632f4e8c9ba4d7dadf62ff5c41d43f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:56.067263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:56.068299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-43810bb3-0224-4656-a0dd-93da9beba04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:56.071597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 836/1699] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:55 2022] DELETE /identity/v3/projects/8632f4e8c9ba4d7dadf62ff5c41d43f5 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:56.071730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 864/1700] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:55 2022] PUT /identity/v3/projects/917d00134c234e7695460ce39ad9360a/users/3d3a9ebe90a34b389d16eb3e01ca045c/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:56.085630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860306-1bf6-44d5-bbdd-aeab96ca933c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.085818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.086285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.086495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860306-1bf6-44d5-bbdd-aeab96ca933c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.086752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.087082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860306-1bf6-44d5-bbdd-aeab96ca933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.120560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b3860306-1bf6-44d5-bbdd-aeab96ca933c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:56.124542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b3860306-1bf6-44d5-bbdd-aeab96ca933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.162495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] MFA Rules not processed for user `3d3a9ebe90a34b389d16eb3e01ca045c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:56.165868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.248150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 837/1701] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:56.257054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.257422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.258972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.310030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:56.314858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.343234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 838/1702] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:56.352574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:56.366846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 865/1703] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:56.369830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:56.373565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.397239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a8d07-0603-4104-a5ef-b121d9181b1f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:56.417252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266258d-ebaf-4959-b127-324ba32e0b4f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:56.418762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:56.422281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.428706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:56.429305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.430043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.430281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.430628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.431057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.431949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:56.432680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:56.434194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:56.434793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:56.440362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:56.476215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 839/1704] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:56.477558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:56.483054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.609159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 866/1705] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:56 2022] POST /identity/v3/projects => generated 369 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:56.612145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:56.759602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea8c43-aea1-414d-b1b7-8a0935bf965e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:56.785461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cc5c1-c45c-4a28-9326-bc9ead91a24c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:56.801142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:56.801778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.802293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.803010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.803207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:56.805624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:56.864017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:56.866515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:56.866943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:56.868211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:56.870854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.949320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 867/1706] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:56 2022] POST /identity/v3/projects => generated 387 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:56.949320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:56.952109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:56.979503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:56.981379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900d580-4668-42c7-b64d-e9fe8241b143 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:56.995475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 840/1707] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:45:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:45:57.013734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:57.014110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:57.014456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:57.067487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] MFA Rules not processed for user `d614a216aa0c42a983af191117871a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:57.067487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:57.116531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50baf51-2c5d-4611-abfb-0e497411e41e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:57.134365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:57.137720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:57.138048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:57.141016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] PATH_INFO: `/v3/users/6f7d6dbe27ba42169a0f26d58c68f50c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:57.147052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7d6dbe27ba42169a0f26d58c68f50c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:57.162915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:57.212188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:57.389366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 841/1708] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:57.389366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:57.421197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9288dd-480e-42cb-b48e-c7c0eeff9fac None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:57.452488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:57.458376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:57.458376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:57.458376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:57.458376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:57.463828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:57.478873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7d6dbe27ba42169a0f26d58c68f50c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:57.512159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7d6dbe27ba42169a0f26d58c68f50c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:57.533028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7d6dbe27ba42169a0f26d58c68f50c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:57.533028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7d6dbe27ba42169a0f26d58c68f50c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:57.657517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 842/1709] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:57 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:57.661472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:57.679973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:45:57.681334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7d6dbe27ba42169a0f26d58c68f50c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:57.688689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 868/1710] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:45:56 2022] DELETE /identity/v3/users/6f7d6dbe27ba42169a0f26d58c68f50c => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:57.696128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:57.696128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:57.720322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258d1f-65bb-45fe-8b3c-73e86c5e81c6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:57.949292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e85d-ddae-4661-96ba-71e2e45fd9d5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:57.973384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:57.974044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:57.974357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:57.974637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] PATH_INFO: `/v3/projects/c7dd050f40b8417cb0a00f672d629f80` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:57.977506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7dd050f40b8417cb0a00f672d629f80)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:57.979633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:57.984683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:57.987637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:57.987637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:57.987637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:57.988746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:57.991599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ac53-a079-498e-9efb-54f9b2af0196 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:58.178487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 843/1711] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:45:57 2022] POST /identity/v3/users => generated 294 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:58.181440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:58.182032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:58.182853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:58.278646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] MFA Rules not processed for user `d614a216aa0c42a983af191117871a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:58.302296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:58.331793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7dd050f40b8417cb0a00f672d629f80', 'request_id': 'req-6d77ee3b-9f47-4234-be56-5fb73d50eb42'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:58.350572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7dd050f40b8417cb0a00f672d629f80', 'request_id': 'req-6d77ee3b-9f47-4234-be56-5fb73d50eb42'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:58.398703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7dd050f40b8417cb0a00f672d629f80', 'request_id': 'req-6d77ee3b-9f47-4234-be56-5fb73d50eb42'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:58.471644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 844/1712] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:58.472555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:58.478734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:58.543920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] The token cache is being invalidate because project c7dd050f40b8417cb0a00f672d629f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:45:58.544713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:58.546112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d77ee3b-9f47-4234-be56-5fb73d50eb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:45:58.560716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 869/1713] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:45:57 2022] DELETE /identity/v3/projects/c7dd050f40b8417cb0a00f672d629f80 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:58.563309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907950dc-0318-4a92-9d18-658a311557e0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:58.564045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907950dc-0318-4a92-9d18-658a311557e0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:58.564045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907950dc-0318-4a92-9d18-658a311557e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:58.641247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-907950dc-0318-4a92-9d18-658a311557e0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:58.645869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-907950dc-0318-4a92-9d18-658a311557e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.103871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec274d1-2e93-4bbd-903e-57005a99fb45 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.103871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.103871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.103871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 845/1714] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.106185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.109179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 870/1715] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:59.109179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97753-d549-47fc-afdb-8db6a1865104 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.109179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97753-d549-47fc-afdb-8db6a1865104 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.109179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97753-d549-47fc-afdb-8db6a1865104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.110585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45389f-4e61-4c77-b982-4805dfc96eb8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.111386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-05c97753-d549-47fc-afdb-8db6a1865104 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:59.111386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-05c97753-d549-47fc-afdb-8db6a1865104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.112842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.112842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.112842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.112842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.114287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.114287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.114287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 846/1716] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:59.114287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.114287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 871/1717] 10.222.0.61 () {58 vars in 1015 bytes} [Fri Sep 9 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:59.115401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.116560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 872/1718] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:59.116560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.116560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.116560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204688d-32a6-4096-9e14-1228d831cf46 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.117230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458202-3426-4976-a1ae-2137458eb368 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.117380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.117380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.117380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.117380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.117964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.117964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.117964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.117964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.118557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.118715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.118856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.119003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.119182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 873/1719] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:59.119336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.119336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 847/1720] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:59.119663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.119822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.120013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.134177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95d9e-0080-48e6-852c-8036f6dd179d service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.145466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eda9e1-ef95-4f12-9963-36f2e5c202f8 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.151181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.152675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.153146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.153649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] PATH_INFO: `/v3/projects/d7fac6f013d94c59801313e73ae2f3c0/users/2f87dcabc72a4e66b489b5b8e2fc888d/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.153761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.154455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.154827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.155947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.156907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.158873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e97fab-fc96-4340-8866-3032b5a6a6c7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.178459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 848/1721] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:59.180584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.180948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.181473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.182212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fac6f013d94c59801313e73ae2f3c0, user_id=2f87dcabc72a4e66b489b5b8e2fc888d, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.185649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.214322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:59.215998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.238252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 849/1722] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:45:59.239131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.241459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:45:59.272895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048b08-71d6-4de2-bd81-42a2d86295cb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.282350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.283127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.283681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.284215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.314287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.315883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.331666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 850/1723] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:45:59.332970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.370460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 874/1724] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:59 2022] PUT /identity/v3/projects/d7fac6f013d94c59801313e73ae2f3c0/users/2f87dcabc72a4e66b489b5b8e2fc888d/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:59.379202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.396418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3efd5a-e53d-4227-891f-7ce1a66d6b91 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.563616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431b837-2fd2-48e9-a152-df9b32bac11f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.582589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.583221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.583504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.583769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.585025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:45:59.585025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.591102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.591333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.591333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.591333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.591333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] PATH_INFO: `/v3/projects/44de37fb2d3d4d92b593b8cdfa910350/users/94f643debda34362b973da71fa422d06/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.609658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 875/1725] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:45:59.610466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.634794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4d8d-ef84-4b95-9658-75438c51a570 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.660173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.660173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.660173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.660173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.661008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.661008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.673137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44de37fb2d3d4d92b593b8cdfa910350, user_id=94f643debda34362b973da71fa422d06, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.676864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.685502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 876/1726] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:45:59.686377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.704297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997704cf-fc21-4ed3-855e-c42b909a3023 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:45:59.733811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:45:59.733811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.733811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.733811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.734248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:45:59.734248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c73483-1173-455c-9cbc-400597dd2021 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:45:59.809693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 851/1727] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:59 2022] PUT /identity/v3/projects/44de37fb2d3d4d92b593b8cdfa910350/users/94f643debda34362b973da71fa422d06/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:45:59.822697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:45:59.850087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 877/1728] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:45:59 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:45:59.856267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:45:59.856267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:45:59.856267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:45:59.931219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] MFA Rules not processed for user `94f643debda34362b973da71fa422d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:45:59.934827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:00.090734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3976-58e9-4479-ae51-7a59c9ce1b87 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:00.266927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 878/1729] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:00.275692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:00.444819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:00.445578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:00.449438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:00.449748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] PATH_INFO: `/v3/projects/d7fac6f013d94c59801313e73ae2f3c0/users/2f87dcabc72a4e66b489b5b8e2fc888d/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:00.455112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fac6f013d94c59801313e73ae2f3c0, user_id=2f87dcabc72a4e66b489b5b8e2fc888d, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:00.472629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:00.487353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:00.540667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 852/1730] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:45:59 2022] PUT /identity/v3/projects/d7fac6f013d94c59801313e73ae2f3c0/users/2f87dcabc72a4e66b489b5b8e2fc888d/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:00.541444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f282bc-7dfe-4643-b50c-130e5259643a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:00.543823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:00.565984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:00.750254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:00.750906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:00.751182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:00.751450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:00.752100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:00.766871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57786-b06b-4e82-9285-970ef6127254 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:00.829099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609c866-d0ca-429b-a995-66ae9c651b38 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:00.848653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:00.848653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:00.848653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:00.848653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:00.848985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:00.852880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:00.913193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 879/1731] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:00 2022] POST /identity/v3/users => generated 293 bytes in 638 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:00.915115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:00.915387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:00.917840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:00.990370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] MFA Rules not processed for user `2f87dcabc72a4e66b489b5b8e2fc888d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:00.991957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:01.052888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 853/1732] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:00 2022] POST /identity/v3/projects => generated 387 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:01.053778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.203315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 880/1733] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:01.208318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.256921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d30a4d-f589-4bee-9b1e-e1b3b05cc568 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.274536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275e7d8-86ff-43a3-86a4-a595f2cc7688 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.279267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.280003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.280290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.280563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.281223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.284659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.290601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:01.295649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.301536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.301536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.301536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.306430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.324097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.329698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 854/1734] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:01.329698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.338782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:01.445604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 881/1735] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:01 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:01.449728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.489340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1487f7-4cfd-43af-90d1-5955085fffdc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.513672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.517091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.517598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.517918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.536361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.546405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.548071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c6d0-233e-4428-b503-7a6c2afce8db admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.556901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.560399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.560399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.560399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.591599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 882/1736] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:01.592927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.648316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64aa4c-db08-41cb-af07-2d234401f908 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.658989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.660743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.662013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.662013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.662759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.665563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.681373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 883/1737] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:01.681373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.684590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.686025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.701480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb832-7d94-42f5-8d1d-3a02c4a2aac7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.721131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.722035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.727213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.727213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] PATH_INFO: `/v3/projects/3cf05fb1efd943c89fef84b661fa80d9/users/99d4adc8e7e2489ba05f7425104f9281/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.727539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 855/1738] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:01.729596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:01.763994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08712b6-b5ec-42d1-8653-bd52bba8c01f service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:01.780984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:01.784579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:01.784579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:01.784579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] PATH_INFO: `/v3/projects/359036683b1f4aae8ee4a69a03f3da7f/users/8b405d33f5f745ad9dacacb1056c6640/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:01.784845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf05fb1efd943c89fef84b661fa80d9, user_id=99d4adc8e7e2489ba05f7425104f9281, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.786512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d8d72-7981-4f83-852b-0d484fd7c574 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.810576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359036683b1f4aae8ee4a69a03f3da7f, user_id=8b405d33f5f745ad9dacacb1056c6640, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:01.813708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:01.984257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 856/1739] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:01 2022] PUT /identity/v3/projects/359036683b1f4aae8ee4a69a03f3da7f/users/8b405d33f5f745ad9dacacb1056c6640/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:01.993081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 884/1740] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:01 2022] PUT /identity/v3/projects/3cf05fb1efd943c89fef84b661fa80d9/users/99d4adc8e7e2489ba05f7425104f9281/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:02.035026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.038310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.040037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:02.041280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.045110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:02.119022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] MFA Rules not processed for user `99d4adc8e7e2489ba05f7425104f9281`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:02.122599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:02.298249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a4ed-6f9e-4e21-8d74-11b9abad7b27 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:02.314276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:02.317931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.320759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.327374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.327374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:02.334731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:02.350750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 857/1741] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:02.352522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:02.425289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 885/1742] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:02.425289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:02.427197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:02.461183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada446d3-90dd-4e26-b5c0-ece38d8bd707 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:02.501569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:02.501569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.501569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.501569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] PATH_INFO: `/v3/projects/267962bfc8234bbeac2bebfa3e36d3fa` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.501883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] RBAC: Authorizing `identity:delete_project(project_id=267962bfc8234bbeac2bebfa3e36d3fa)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:02.501883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:02.503866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26453ab4-45e7-455a-985b-a5e1c0471391 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:02.508772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:02.509478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.510667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.511148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.513509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:02.671841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:02.673383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:02.686399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 858/1743] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:02.687254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:02.705862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812245ce-88b7-411b-92fc-fbbd57970e4b service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:02.706531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:02.707504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.707796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.708467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] PATH_INFO: `/v3/projects/359036683b1f4aae8ee4a69a03f3da7f/users/8b405d33f5f745ad9dacacb1056c6640/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.711381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359036683b1f4aae8ee4a69a03f3da7f, user_id=8b405d33f5f745ad9dacacb1056c6640, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:02.715505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:02.738889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 859/1744] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:02 2022] PUT /identity/v3/projects/359036683b1f4aae8ee4a69a03f3da7f/users/8b405d33f5f745ad9dacacb1056c6640/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:02.738889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:02.742419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267962bfc8234bbeac2bebfa3e36d3fa', 'request_id': 'req-83144953-1a48-4ed4-8d00-914ac504577c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:02.749723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267962bfc8234bbeac2bebfa3e36d3fa', 'request_id': 'req-83144953-1a48-4ed4-8d00-914ac504577c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:02.777121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267962bfc8234bbeac2bebfa3e36d3fa', 'request_id': 'req-83144953-1a48-4ed4-8d00-914ac504577c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:02.880168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] The token cache is being invalidate because project 267962bfc8234bbeac2bebfa3e36d3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:02.880478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:02.881426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-83144953-1a48-4ed4-8d00-914ac504577c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:02.889558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 886/1745] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:02 2022] DELETE /identity/v3/projects/267962bfc8234bbeac2bebfa3e36d3fa => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:02.891017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.891475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.891873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.943932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f4856-bc87-4a45-a7c1-270f085d0086 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:02.951427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:02.952311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:02.952756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:02.953173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:02.954203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:02.957371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9571b6-6cc8-4ffa-a52f-3ae882bffcd3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:02.997344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] MFA Rules not processed for user `8b405d33f5f745ad9dacacb1056c6640`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:02.999079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:03.105633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 860/1746] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:02 2022] POST /identity/v3/projects => generated 383 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:03.119095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.120816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.123896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.230018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:03.239910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:03.282483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 887/1747] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:03.287721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:03.289600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:03.330672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a56c71-0619-4c8d-b68d-94089474c1df None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:03.443440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 861/1748] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:03.444365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:03.446857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:03.555365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:03.560946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.563598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.563598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.565327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:03.575133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59b50d-048c-4603-9be8-7234c5fd9d8e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:03.620486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af04cfa-f987-4218-92a2-bb0a2561a3d9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:03.633733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:03.634017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.639880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.640213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.764641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 888/1749] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:03 2022] POST /identity/v3/users => generated 292 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:03.766464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77636815-9949-46f6-9790-fcd42da44bf0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.771058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77636815-9949-46f6-9790-fcd42da44bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.771400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77636815-9949-46f6-9790-fcd42da44bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.830607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-77636815-9949-46f6-9790-fcd42da44bf0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:03.833082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-77636815-9949-46f6-9790-fcd42da44bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:03.843876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 889/1750] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:03.851270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50365-2c41-42c2-a752-12934d2e94c9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.851697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50365-2c41-42c2-a752-12934d2e94c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.852012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50365-2c41-42c2-a752-12934d2e94c9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.860894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 890/1751] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:46:03 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:46:03.861944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50365-2c41-42c2-a752-12934d2e94c9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:03.918697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:03.923951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:03.924725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50365-2c41-42c2-a752-12934d2e94c9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:03.946551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:03.947231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:03.947513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:03.947794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:03.948459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:03.950134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:03.961181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 862/1752] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:03.990407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:04.003412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 891/1753] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:04.019322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aafd5-50ee-48f1-aa76-05211a83c9ed service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:04.026114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:04.028512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:04.029172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.029452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.031385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.032743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:04.033073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:04.056287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd443e-0bc2-433a-891f-acdb9d773c44 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:04.073146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73fe83-5fee-4e04-931d-7900404a57f3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:04.115767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:04.117165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.117498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.117778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.118477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:04.121724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:04.145534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 863/1754] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:04.146915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.154553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.155172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.175976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 892/1755] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:04.176605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:04.207276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:04.208581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:04.232553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572d97-c57e-442f-b92a-9102e1aacc31 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:04.254395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:04.256433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.256820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.259361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.260180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:04.266151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:04.434842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 893/1756] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:04 2022] POST /identity/v3/projects => generated 373 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:04.435604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:04.444793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 864/1757] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:04.445671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:04.447937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:04.495533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304709c3-ced6-462d-b1bc-60dd7c712ed2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:04.501030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:04.602417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:04.604391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.604886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.605556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] PATH_INFO: `/v3/users/94f643debda34362b973da71fa422d06` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.607155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f643debda34362b973da71fa422d06)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:04.609750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:04.670890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ae37-9951-4080-a44a-ffc9143a12f2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:04.680811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:04.682449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:04.683473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:04.684217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] PATH_INFO: `/v3/projects/cefcb6e7201346e393b7cb616d6a15d8/users/775d82b6b32c486bb623aa3defb853b5/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:04.708955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cefcb6e7201346e393b7cb616d6a15d8, user_id=775d82b6b32c486bb623aa3defb853b5, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:04.713133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:04.967738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 894/1758] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:04 2022] PUT /identity/v3/projects/cefcb6e7201346e393b7cb616d6a15d8/users/775d82b6b32c486bb623aa3defb853b5/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:04.969181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:05.001925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f643debda34362b973da71fa422d06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:05.023953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f643debda34362b973da71fa422d06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:05.044091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f643debda34362b973da71fa422d06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:05.107712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d1b3c-f698-4f9a-afa0-7e5135b40571 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:05.110305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:05.123208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:05.123861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.124146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.124422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.125128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:05.127488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461f544-e496-4618-a886-f5a85f61c1a0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:05.165097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f643debda34362b973da71fa422d06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:05.166034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f643debda34362b973da71fa422d06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:05.177718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 865/1759] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:04 2022] DELETE /identity/v3/users/94f643debda34362b973da71fa422d06 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:05.178772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:05.184007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.305874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb2dbf-e676-4ac8-bf6f-4842db2ff2b0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:05.316086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade849e-4822-4629-8700-a293743a250b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:05.316935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade849e-4822-4629-8700-a293743a250b service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.317211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade849e-4822-4629-8700-a293743a250b service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.317482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade849e-4822-4629-8700-a293743a250b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.320993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7ade849e-4822-4629-8700-a293743a250b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.330684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 895/1760] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:04 2022] POST /identity/v3/users => generated 287 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:05.331964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4b5b-24b9-4dbb-aa77-87f76dc67911 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.332356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4b5b-24b9-4dbb-aa77-87f76dc67911 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.332636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4b5b-24b9-4dbb-aa77-87f76dc67911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.357017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade849e-4822-4629-8700-a293743a250b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:05.362118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade849e-4822-4629-8700-a293743a250b service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:05.404378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c97f4b5b-24b9-4dbb-aa77-87f76dc67911 None None] MFA Rules not processed for user `775d82b6b32c486bb623aa3defb853b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:05.406102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c97f4b5b-24b9-4dbb-aa77-87f76dc67911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.410214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 866/1761] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:05.414093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.414246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.414487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.452793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] MFA Rules not processed for user `e00238b83af948fe8752f998aa08b0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:05.454029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.576157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 896/1762] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:05.577543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171a42-15b0-4c6b-b494-ba7188e35ea2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.577829 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171a42-15b0-4c6b-b494-ba7188e35ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.578361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4171a42-15b0-4c6b-b494-ba7188e35ea2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.579890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 897/1763] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:46:05.580914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.581302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.581586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.610383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 867/1764] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:05.614650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:05.614650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.615646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:05.618218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.720204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 898/1765] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:05.720204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:05.723677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:05.754843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e02c-3ceb-414c-a3bf-f5ca693cb164 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:05.762639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:05.775187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:05.776512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.776800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.777256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.777811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:05.778273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8b4d0-a5a4-4a88-b4b1-4ad9edf913a2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:05.779885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef361e97-fd73-49ac-9694-f8d6db2cfad5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:05.823408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:05.827344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.829774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.830159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.833636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:05.870316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bc409-4820-4997-a7ea-7c16bc520523 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:05.915233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 899/1766] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:05.935142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 868/1767] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:05.942483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.942979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.943363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:05.947899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:05.948349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:05.948631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.003190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] MFA Rules not processed for user `683fe1045ddb4bc1a8a94d3948495f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:06.005187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:06.006387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:06.008647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:06.171648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 869/1768] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:06.173067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.186955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:06.252507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 900/1769] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:06.253571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.300415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293ea4f-46f9-43ad-9db7-145d4e62d768 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.338147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.345169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.345169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.345169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.518077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d7b1-bf00-4e28-9083-ea35db75b0da None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.544567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.545216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.548227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.548227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.548926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:06.550877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:06.655157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:06.656668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:06.670100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 870/1770] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:06.671096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.700648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42d10d-aeaa-4026-a644-7c1b1e9844c9 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.712671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.713860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.714754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.715220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.720027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:06.723083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd38aa9-4304-48e5-a0bd-cf7997b40ed5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:06.776637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 871/1771] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:06.778763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5a07-6368-46d6-a49c-0cbf495fccbd None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.779275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5a07-6368-46d6-a49c-0cbf495fccbd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.779843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5a07-6368-46d6-a49c-0cbf495fccbd None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.789145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 901/1772] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:06 2022] POST /identity/v3/users => generated 299 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:06.792271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.808297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 872/1773] 10.222.0.60 () {54 vars in 912 bytes} [Fri Sep 9 11:46:06 2022] GET /identity => generated 270 bytes in 30 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:46:06.809314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5a07-6368-46d6-a49c-0cbf495fccbd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.829932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd7ac-9bcc-48ee-b602-2228d746cb30 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.835856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.836096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5a07-6368-46d6-a49c-0cbf495fccbd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.836580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.836972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.837358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] PATH_INFO: `/v3/projects/998afeb5b77b46ddb66379a2eb681b07/users/17e27d5412ba4aa4899712e9b7b124e1/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.840966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.841613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.841911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.842253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.843110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:06.844930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295172d8-64d4-401d-a101-1be9c05597a0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:06.861347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 873/1774] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:06.861347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.861347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.861347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:06.867769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=998afeb5b77b46ddb66379a2eb681b07, user_id=17e27d5412ba4aa4899712e9b7b124e1, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:06.871775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:06.920554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:06.922168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:06.938651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 874/1775] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:06.938856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:06.952142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea5c22-fff2-42b9-86fc-88fa7745b419 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:06.958816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:06.959561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:06.962326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:06.962326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] PATH_INFO: `/v3/projects/c3f58efab891462bab20dc053b873910/users/b7123c0813b34f3c9b467630f6884ea5/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.006256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f58efab891462bab20dc053b873910, user_id=b7123c0813b34f3c9b467630f6884ea5, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.011215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10488-6cea-4510-ba78-247d8b2d579f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:07.076374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 902/1776] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:06 2022] PUT /identity/v3/projects/998afeb5b77b46ddb66379a2eb681b07/users/17e27d5412ba4aa4899712e9b7b124e1/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:07.076374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:07.078379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:07.134999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 875/1777] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:06 2022] PUT /identity/v3/projects/c3f58efab891462bab20dc053b873910/users/b7123c0813b34f3c9b467630f6884ea5/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:07.136768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.137120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.137527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.215596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c5d2b-fe27-4a9d-addc-16d3698f19b1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:07.243757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] MFA Rules not processed for user `17e27d5412ba4aa4899712e9b7b124e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:07.243757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:07.320819 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:07.322133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.322439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.322784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.333014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.334752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:07.347386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 903/1778] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:07.348676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:07.501834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 876/1779] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:07.507429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:07.510913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ad96-46ea-4f9e-87ca-a84276a1cf63 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:07.525667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:07.527844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.528136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.530924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.534031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.536072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:07.555754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c9cfb-ff87-4034-b56c-fbd07b67237c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:07.561574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:07.562282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.562585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.562882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.563605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.567582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:07.568200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 904/1780] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:07.569850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:07.606699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a27b39-e370-42fa-8cdc-6c9600653632 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:07.614532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:07.617447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.617760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.618032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] PATH_INFO: `/v3/projects/c3f58efab891462bab20dc053b873910/users/b7123c0813b34f3c9b467630f6884ea5/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.623598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f58efab891462bab20dc053b873910, user_id=b7123c0813b34f3c9b467630f6884ea5, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.632357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee8e68-0541-40a7-b7f6-3f22beae9a71 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:07.684708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 905/1781] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:07 2022] PUT /identity/v3/projects/c3f58efab891462bab20dc053b873910/users/b7123c0813b34f3c9b467630f6884ea5/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:07.697921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.698336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.698683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.724695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 877/1782] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:07 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:07.729673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:07.819138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] MFA Rules not processed for user `b7123c0813b34f3c9b467630f6884ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:07.820973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:07.937764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c62a4-3b26-4364-8288-d391553e9786 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:07.951562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:07.952712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:07.953028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:07.953313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:07.953967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:07.962281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.104686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 878/1783] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:07 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:08.109037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:08.140618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c488806-fcbc-48d4-9c62-d61d2e834dcf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:08.165657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:08.166163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.166451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.166730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.167452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:08.170621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 906/1784] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:08.171686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:08.173183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.177054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:08.200382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 879/1785] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:08.209353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:08.237584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c82c55-279f-4878-933d-759210f47ea2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:08.250813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:08.256369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.257263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.260015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] PATH_INFO: `/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b/users/bd66504b7bfe46a2bf2a562077b0a637/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.306495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:08.319038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2030e1f9dc9b4b8da90263fc3c51797b, user_id=bd66504b7bfe46a2bf2a562077b0a637, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:08.320119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71f8e6-f244-438b-91d1-0cb0eb722d8e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:08.324733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:08.325230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.325764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.326319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.326789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.329236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:08.429865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 880/1786] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:08 2022] PUT /identity/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b/users/bd66504b7bfe46a2bf2a562077b0a637/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:08.436907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:08.500609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:08.505093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.548548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 907/1787] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:08.591749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:08.615103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbb8bd-accb-4e62-93dc-c4f4495cc3fc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:08.621172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:08.622236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.622679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.623221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.626009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:08.628139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa298f-b66c-4fda-bfe5-39d3be6e14c7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.641970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 881/1788] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:08.645871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:08.664255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b14b1-1021-4401-84a4-1addf488d61d service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:08.699233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:08.700083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.702941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.703279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] PATH_INFO: `/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b/users/bd66504b7bfe46a2bf2a562077b0a637/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.713341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2030e1f9dc9b4b8da90263fc3c51797b, user_id=bd66504b7bfe46a2bf2a562077b0a637, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:08.718241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:08.736925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 908/1789] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:08 2022] PUT /identity/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b/users/bd66504b7bfe46a2bf2a562077b0a637/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:08.748874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:08.749427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:08.749951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:08.831735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] MFA Rules not processed for user `bd66504b7bfe46a2bf2a562077b0a637`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:08.834798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:09.081378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 882/1790] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:09.415410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:09.417407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:09.424380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:09.510690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:09.559051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6d7be-633a-48dc-aa5a-fa4d9b00092a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:09.576125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:09.577031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:09.577524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:09.578084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:09.609814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:09.611855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:09.612901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133034bc-e381-4fe4-ba0e-7254b3ab9c42 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:09.621037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:09.628274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:09.628274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:09.628274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] PATH_INFO: `/v3/projects/44de37fb2d3d4d92b593b8cdfa910350` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:09.628274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44de37fb2d3d4d92b593b8cdfa910350)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:09.628274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:09.641974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 883/1791] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:09.875144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:09.920601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44de37fb2d3d4d92b593b8cdfa910350', 'request_id': 'req-020518e3-4fe6-4b2a-a0bd-d28a55773f48'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:09.947583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44de37fb2d3d4d92b593b8cdfa910350', 'request_id': 'req-020518e3-4fe6-4b2a-a0bd-d28a55773f48'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:09.979256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44de37fb2d3d4d92b593b8cdfa910350', 'request_id': 'req-020518e3-4fe6-4b2a-a0bd-d28a55773f48'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.070047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63321-8511-4f89-bc88-90c86bbf955d service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:10.095568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] The token cache is being invalidate because project 44de37fb2d3d4d92b593b8cdfa910350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:10.096198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.097330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.099685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 909/1792] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:09 2022] DELETE /identity/v3/projects/44de37fb2d3d4d92b593b8cdfa910350 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:10.107183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:10.118315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:10.218966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:10.219959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:10.220528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:10.221029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:10.228825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:10.384602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020518e3-4fe6-4b2a-a0bd-d28a55773f48 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:10.403062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:10.404939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:10.415208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:10.421593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:10.423345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:10.423939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] PATH_INFO: `/v3/users/8b405d33f5f745ad9dacacb1056c6640` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:10.425952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b405d33f5f745ad9dacacb1056c6640)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:10.429302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 884/1793] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:10.432877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:10.738888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b405d33f5f745ad9dacacb1056c6640'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.768891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b405d33f5f745ad9dacacb1056c6640'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.783471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b405d33f5f745ad9dacacb1056c6640'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.784612 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b405d33f5f745ad9dacacb1056c6640'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.940285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b405d33f5f745ad9dacacb1056c6640'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:10.958726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 910/1794] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:10 2022] DELETE /identity/v3/users/8b405d33f5f745ad9dacacb1056c6640 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:11.275692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:11.539885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:11.552104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f681b0-4ae7-447a-ad32-54a0a6e73780 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:11.575497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:11.617157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:11.617789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:11.618100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:11.618380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] PATH_INFO: `/v3/projects/359036683b1f4aae8ee4a69a03f3da7f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:11.619581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=359036683b1f4aae8ee4a69a03f3da7f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:11.626810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:11.778106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359036683b1f4aae8ee4a69a03f3da7f', 'request_id': 'req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:11.872202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359036683b1f4aae8ee4a69a03f3da7f', 'request_id': 'req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:11.879318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359036683b1f4aae8ee4a69a03f3da7f', 'request_id': 'req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:11.880611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:11.882255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:11.891323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] The token cache is being invalidate because project 359036683b1f4aae8ee4a69a03f3da7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:11.891620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:11.892313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8c34a4ea-41e1-4a7f-8972-dbe07a3c0a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:11.893843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 885/1795] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:11 2022] DELETE /identity/v3/projects/359036683b1f4aae8ee4a69a03f3da7f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:11.895540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e545-70bf-4481-a312-932b9564124d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:11.898866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b2358-bd27-4665-bafd-d8bdf9dbc9c7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:11.899149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b2358-bd27-4665-bafd-d8bdf9dbc9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:11.899500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b2358-bd27-4665-bafd-d8bdf9dbc9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:11.919289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bb5b2358-bd27-4665-bafd-d8bdf9dbc9c7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:11.920762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bb5b2358-bd27-4665-bafd-d8bdf9dbc9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:11.954438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:11.955497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:11.955926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:11.956347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] PATH_INFO: `/v3/users/17e27d5412ba4aa4899712e9b7b124e1` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:11.957940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e27d5412ba4aa4899712e9b7b124e1)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:11.962774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:11.988007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 886/1796] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:11.996300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:11.996589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:11.996866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.015948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:12.017153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:12.025090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 887/1797] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:12.029063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.039703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1ba15-97a0-4e5c-8fb4-379ea7e63629 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.044232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.044836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.045113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.045382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.046013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:12.046341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.047925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.059995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 888/1798] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:46:12.068570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.077857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77bee1-cb82-4acd-8717-45728847a64d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.082525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.083095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.083354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.083624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.084235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.086446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.262143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 889/1799] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:12 2022] POST /identity/v3/projects => generated 401 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:12.264673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.288303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e27d5412ba4aa4899712e9b7b124e1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:12.304421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e27d5412ba4aa4899712e9b7b124e1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:12.313286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e27d5412ba4aa4899712e9b7b124e1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:12.314003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e27d5412ba4aa4899712e9b7b124e1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:12.342703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227887b1-84ec-4149-96a8-336a04521e24 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.347554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.348220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.348501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.348776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.349417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.351469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.394808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-788625a1-2bf8-4d2a-bff3-a20e9d56060d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e27d5412ba4aa4899712e9b7b124e1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:12.401356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 911/1800] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:11 2022] DELETE /identity/v3/users/17e27d5412ba4aa4899712e9b7b124e1 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:12.405599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.405887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.406237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.426656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:12.427878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:12.490136 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 912/1801] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:12.490283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 890/1802] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:12 2022] POST /identity/v3/users => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:12.492482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.493863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:12.496603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.500766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:12.505658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453d0c2-bb04-4d9f-a494-57cf9cdf2c3a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.510832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.511547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.512075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.512440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.513555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.515374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.529942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 913/1803] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:46:12.530977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.541372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e19ed-696b-4057-adb4-512bdecb328e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.546422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.547115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.547482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.547828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.548538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.550239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.560574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 914/1804] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:12.561355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83b78-1aa6-4310-8948-81b7b2137bef admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.564917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.575407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc14e3-2b6b-428d-8f12-e1ec611e6f8f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.580164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.580919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.581421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.581859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] PATH_INFO: `/v3/projects/c3bb3f6ece8247efb445857ff4693a9d/users/4c378cacfb5c445bb97719b40d052a89/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.597518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bb3f6ece8247efb445857ff4693a9d, user_id=4c378cacfb5c445bb97719b40d052a89, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.600785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.617243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.617889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.618170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.618442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.627549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.629019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3e469-5008-4b81-a899-15ab0c5326f0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:12.639028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 891/1805] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:12.680525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 915/1806] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:12 2022] PUT /identity/v3/projects/c3bb3f6ece8247efb445857ff4693a9d/users/4c378cacfb5c445bb97719b40d052a89/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:12.691847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.692132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.692450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.721741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] MFA Rules not processed for user `4c378cacfb5c445bb97719b40d052a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:12.723110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:12.789287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 892/1807] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:12.793745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:12.874416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ba0-50bf-444f-a2ae-88723a9b080a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:12.880576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:12.882026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:12.882649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:12.882884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:12.883635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:12.885671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.087918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 916/1808] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:12 2022] POST /identity/v3/projects => generated 399 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:13.092268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.103690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b99ee1-ad19-46be-ab77-1aea8e2b704d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.164134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.164753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.165035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.165300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.165931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.167891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.182952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.183864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:13.198270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d8b80-f32d-4bec-8905-b5518f42089d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.203108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.203843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.204199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.205747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] PATH_INFO: `/v3/projects/998afeb5b77b46ddb66379a2eb681b07` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.206976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=998afeb5b77b46ddb66379a2eb681b07)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.210344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.284530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 893/1809] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:13 2022] POST /identity/v3/users => generated 306 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:13.289035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.301231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db435623-6744-41a4-a188-9ea9661268b2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.306790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.307388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.307775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.308051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.308698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.310354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.311391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:13.322656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 894/1810] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:13.326970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.337023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f00c-3fb7-435e-ac07-917498f2a569 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.342107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.343424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.343424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.343424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] PATH_INFO: `/v3/projects/bbf05684fcd942f99a52562ecada3cb3/users/3433e228b0ba42e98feab2347787ccb7/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.355624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998afeb5b77b46ddb66379a2eb681b07', 'request_id': 'req-016dfe0d-0700-4d17-b463-64cad6102839'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.358618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf05684fcd942f99a52562ecada3cb3, user_id=3433e228b0ba42e98feab2347787ccb7, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.361366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.362471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998afeb5b77b46ddb66379a2eb681b07', 'request_id': 'req-016dfe0d-0700-4d17-b463-64cad6102839'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.369415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998afeb5b77b46ddb66379a2eb681b07', 'request_id': 'req-016dfe0d-0700-4d17-b463-64cad6102839'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.456956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] The token cache is being invalidate because project 998afeb5b77b46ddb66379a2eb681b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:13.457565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.458246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.460255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 917/1812] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:13 2022] DELETE /identity/v3/projects/998afeb5b77b46ddb66379a2eb681b07 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:13.463817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 895/1812] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:13 2022] PUT /identity/v3/projects/bbf05684fcd942f99a52562ecada3cb3/users/3433e228b0ba42e98feab2347787ccb7/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:13.463933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.465384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:13.466440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.468159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:13.518434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:13.536832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414596dd-3b85-4c96-b25e-f54827717b69 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.538825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dfe0d-0700-4d17-b463-64cad6102839 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.542834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.543635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.544302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.544583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.544860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] PATH_INFO: `/v3/users/bd66504b7bfe46a2bf2a562077b0a637` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.546041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd66504b7bfe46a2bf2a562077b0a637)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.546766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.547120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.547511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.547978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.548442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.550363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.565058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 896/1813] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:46:13.566320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.579859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2c311-cc66-41f8-aed7-1cf0460efc77 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.584981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.585637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.585915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.586218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] PATH_INFO: `/v3/projects/bbf05684fcd942f99a52562ecada3cb3/users/3433e228b0ba42e98feab2347787ccb7/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.588222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf05684fcd942f99a52562ecada3cb3, user_id=3433e228b0ba42e98feab2347787ccb7, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.590932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e755b08-f93d-4906-9ba3-a11abc5e0c19 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.602687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 897/1814] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:13 2022] PUT /identity/v3/projects/bbf05684fcd942f99a52562ecada3cb3/users/3433e228b0ba42e98feab2347787ccb7/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:13.612560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147103-6271-4d15-8e58-279e53b03699 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.612841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147103-6271-4d15-8e58-279e53b03699 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.613110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147103-6271-4d15-8e58-279e53b03699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.643950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-88147103-6271-4d15-8e58-279e53b03699 None None] MFA Rules not processed for user `3433e228b0ba42e98feab2347787ccb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:13.645182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-88147103-6271-4d15-8e58-279e53b03699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:13.714510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 898/1815] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:13.727637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147103-6271-4d15-8e58-279e53b03699 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:13.729023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-88147103-6271-4d15-8e58-279e53b03699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:13.784433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147103-6271-4d15-8e58-279e53b03699 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:13.788573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:13.789176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:13.789452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:13.789720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:13.798512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:13.799930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:13.809380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 899/1816] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:13 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:13.855007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd66504b7bfe46a2bf2a562077b0a637'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.867964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd66504b7bfe46a2bf2a562077b0a637'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.874954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd66504b7bfe46a2bf2a562077b0a637'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.875840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd66504b7bfe46a2bf2a562077b0a637'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.969249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd66504b7bfe46a2bf2a562077b0a637'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:13.975978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 918/1817] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:13 2022] DELETE /identity/v3/users/bd66504b7bfe46a2bf2a562077b0a637 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:14.188614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:14.197281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083c7bd-45d2-4092-a66c-10da48859116 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:14.251121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:14.251752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:14.252031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:14.252307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:14.254116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:14.295633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:14.344039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:14.346167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1d164-130c-45b1-944f-8830aa98bb7c service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:14.359477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 900/1818] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:14.491574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:14.562180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:14.563583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:14.629947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28556dc9-504e-4188-a87c-3e73e87ae709 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:14.635062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:14.635696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:14.636159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:14.636444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] PATH_INFO: `/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:14.637609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2030e1f9dc9b4b8da90263fc3c51797b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:14.639626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:14.835910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2030e1f9dc9b4b8da90263fc3c51797b', 'request_id': 'req-fbee0f96-4a91-4d51-9be8-1df9ee39045e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:14.842520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2030e1f9dc9b4b8da90263fc3c51797b', 'request_id': 'req-fbee0f96-4a91-4d51-9be8-1df9ee39045e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:14.848075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2030e1f9dc9b4b8da90263fc3c51797b', 'request_id': 'req-fbee0f96-4a91-4d51-9be8-1df9ee39045e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:14.967037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] The token cache is being invalidate because project 2030e1f9dc9b4b8da90263fc3c51797b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:14.970138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:14.970138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fbee0f96-4a91-4d51-9be8-1df9ee39045e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:14.970138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 919/1819] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:14 2022] DELETE /identity/v3/projects/2030e1f9dc9b4b8da90263fc3c51797b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:14.975444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:14.976172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:14.976172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.010316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:15.011567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:15.091650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 901/1820] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:15.100670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.100961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.101242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.123294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:15.124648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:15.134224 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 920/1821] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:15.138536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.149525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9de87d-b23a-4b24-b148-5dfd6d831178 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.154715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.155424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.155781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.156323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.157081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:15.157460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.159519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.183625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 902/1822] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:46:15.189464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.198634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e33a06-c8ea-4497-8ab5-0aafb5a84e8d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.206514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.207860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.211145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.211145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.211145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.212093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.240387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:15.328262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 921/1823] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:15 2022] POST /identity/v3/projects => generated 407 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:15.333812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.346134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341f863-4802-4e58-8f47-abbd14549676 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.458971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.466980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.467380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.467755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.468570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.470822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.592655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 903/1824] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:15 2022] POST /identity/v3/users => generated 304 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:15.630246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.642800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0107cf-1a0d-4fe4-9905-9887eaa65b2d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.649525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.650435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.650773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.651080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.652146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.654654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.667875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 922/1825] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:15.672414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.683681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7166d3-d195-4067-8cf9-6604a331bbd7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.692165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.692165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.692165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.692165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.692862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.692862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8d559-c293-4b50-bedb-aeac00cb4f35 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.707338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 904/1826] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:15.708842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:15.720350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430384f4-8ae9-4c9a-b23c-f5687accd51f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:15.755349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:15.755349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.755349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.755349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] PATH_INFO: `/v3/projects/53883b89cb56420faf08f6c002710255/users/0fd3fa56688b4e999619ab6c4480faa3/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.757071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53883b89cb56420faf08f6c002710255, user_id=0fd3fa56688b4e999619ab6c4480faa3, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:15.757071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:15.899223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 923/1827] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:15 2022] PUT /identity/v3/projects/53883b89cb56420faf08f6c002710255/users/0fd3fa56688b4e999619ab6c4480faa3/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:15.904898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:15.905205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:15.905476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:15.949175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] MFA Rules not processed for user `0fd3fa56688b4e999619ab6c4480faa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:15.950741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:16.026694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 905/1828] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:15 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:16.032002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.099131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea172fc-1854-4530-9d49-f58e4544b431 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.104529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.105050 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.105332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.105609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.106280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.108208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.332796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 924/1829] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:16 2022] POST /identity/v3/projects => generated 409 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:16.337623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.348200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527040b-ed31-400e-8b9e-6f66e01d9466 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.406688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.407302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.407580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.407851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.408517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.410503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.571635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 906/1830] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:16 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:16.577105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.588193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f50cf-0cff-4683-9d81-ea99895ac897 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.592897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.593525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.593800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.594097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.594766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.596366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.606815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 925/1831] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:16.612981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.623032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46dda-6ed7-4c30-84af-9f554cfc7729 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.627792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.628474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.628759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.629021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] PATH_INFO: `/v3/projects/997e303e088648a0a579dd05fafc2e1c/users/d03636ad79374ea388da66f9a5a5edd2/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.644812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] RBAC: Authorizing `identity:create_grant(project_id=997e303e088648a0a579dd05fafc2e1c, user_id=d03636ad79374ea388da66f9a5a5edd2, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.647594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.780502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:16.787557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 907/1832] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:16 2022] PUT /identity/v3/projects/997e303e088648a0a579dd05fafc2e1c/users/d03636ad79374ea388da66f9a5a5edd2/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:16.791692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.801689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8efb08-648a-42d4-a510-6602b94246dd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.821629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.823326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:16.859347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.859942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.860219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.860491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.861130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.862810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.876828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 926/1833] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:16.877845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:16.879859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699040-e743-453d-84cf-5d54e1c4363d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.889611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.889611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.889611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.889611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.890079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:16.890182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd997b-bb60-40c9-8b04-d58d336e1c6f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:16.892828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:16.893462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.893740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.894046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] PATH_INFO: `/v3/projects/997e303e088648a0a579dd05fafc2e1c/users/d03636ad79374ea388da66f9a5a5edd2/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.895973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997e303e088648a0a579dd05fafc2e1c, user_id=d03636ad79374ea388da66f9a5a5edd2, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.899291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90716a6-8444-4a15-bdff-27f21cc4f525 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:16.910185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 927/1834] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:16 2022] PUT /identity/v3/projects/997e303e088648a0a579dd05fafc2e1c/users/d03636ad79374ea388da66f9a5a5edd2/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:16.924453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:16.924734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:16.925008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:16.962882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] MFA Rules not processed for user `d03636ad79374ea388da66f9a5a5edd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:16.964123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:16.971402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:16.972908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:17.001149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 908/1835] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:17.075951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 928/1836] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:16 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:17.749428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:17.750890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:17.766275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028818a-1535-4d9f-9003-f336a6de0533 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:17.828066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:17.828842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:17.829406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:17.829785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:17.840206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:17.842661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:17.855164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 909/1837] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:17 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:18.546186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:18.596619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:18.607393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0eed-a7ad-4dcc-abfa-895f77345086 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:18.611057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:18.611669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:18.612002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:18.612277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:18.670897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:18.672323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:18.682114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 929/1838] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:18 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:18.745446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:19.685963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:20.202032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:20.211271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee813-dee7-4b7a-83da-931e3a515341 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:20.215089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:20.215875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:20.216229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:20.216591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:20.218450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:20.287472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:20.288901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a3795-1b83-4e2e-9a44-6c6cf4e5610b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:20.298663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 910/1839] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:21.000002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:22.945549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:24.187162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:24.188859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:24.265873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70565a-a383-4332-b05f-68f3522e41d4 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:24.272782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:24.273734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:24.274305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:24.274791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] PATH_INFO: `/v3/users/4c378cacfb5c445bb97719b40d052a89` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:24.276271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c378cacfb5c445bb97719b40d052a89)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:24.278424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:24.524302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c378cacfb5c445bb97719b40d052a89'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:24.536208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c378cacfb5c445bb97719b40d052a89'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:24.544013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c378cacfb5c445bb97719b40d052a89'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:24.544799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c378cacfb5c445bb97719b40d052a89'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:24.613026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c378cacfb5c445bb97719b40d052a89'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:24.619468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 930/1840] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:24 2022] DELETE /identity/v3/users/4c378cacfb5c445bb97719b40d052a89 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:24.624115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:24.624605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:24.624683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:24.644790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:24.646813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:24.723786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 911/1841] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:24.733589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:24.735033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:24.797947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d447191-d4b3-4ecc-95ec-db87c2ef612d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:24.802029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:24.802851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:24.803503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:24.803868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:24.819168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:24.820759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02c4f3-b87a-4466-88be-2c9193e56785 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:24.859031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 931/1842] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:25.227722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:25.375643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:25.377633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:25.403445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cddb18-499b-46f3-a524-7a2c2cd1230b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:25.419182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:25.419840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:25.420118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:25.420389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] PATH_INFO: `/v3/projects/c3bb3f6ece8247efb445857ff4693a9d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:25.422811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bb3f6ece8247efb445857ff4693a9d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:25.424866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:25.469310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa792-8a53-49fc-93ee-4dc40f3db6ae None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:25.469791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa792-8a53-49fc-93ee-4dc40f3db6ae None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:25.470238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa792-8a53-49fc-93ee-4dc40f3db6ae None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:25.472223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 932/1843] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:46:25.490874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:25.491180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:25.491460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:25.519545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:25.521062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:25.609619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bb3f6ece8247efb445857ff4693a9d', 'request_id': 'req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:25.640422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 933/1844] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:25.657893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:25.659356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:25.673573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e4354-1c31-4cb4-bc34-cff6f58a3b01 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:25.747710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bb3f6ece8247efb445857ff4693a9d', 'request_id': 'req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:25.751561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:25.752379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:25.752719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:25.753110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:25.757263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bb3f6ece8247efb445857ff4693a9d', 'request_id': 'req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:25.763717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:25.765404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:25.776496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 934/1845] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:25.778696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] The token cache is being invalidate because project c3bb3f6ece8247efb445857ff4693a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:25.778988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:25.779680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:25.781969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 912/1846] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:25 2022] DELETE /identity/v3/projects/c3bb3f6ece8247efb445857ff4693a9d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:25.794301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:25.870019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c15803-07af-4329-af58-5b51b2d15eb9 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:25.874948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:25.875670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:25.875956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:25.876230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] PATH_INFO: `/v3/users/3433e228b0ba42e98feab2347787ccb7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:25.877359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3433e228b0ba42e98feab2347787ccb7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:25.879301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:26.064569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3433e228b0ba42e98feab2347787ccb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.077401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3433e228b0ba42e98feab2347787ccb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.084982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3433e228b0ba42e98feab2347787ccb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.085686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3433e228b0ba42e98feab2347787ccb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.179409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-81bd615d-e3eb-4386-9acb-af37392bf181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3433e228b0ba42e98feab2347787ccb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.186145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 935/1847] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:25 2022] DELETE /identity/v3/users/3433e228b0ba42e98feab2347787ccb7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:26.539255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:26.666405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:26.667898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:26.685498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f15b-81f0-42b1-af8d-64b9f54e4a8a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:26.761898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:26.762588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:26.762873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:26.763203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] PATH_INFO: `/v3/projects/bbf05684fcd942f99a52562ecada3cb3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:26.765947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf05684fcd942f99a52562ecada3cb3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:26.767932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:26.900381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf05684fcd942f99a52562ecada3cb3', 'request_id': 'req-6afd76bd-3754-4d48-b29e-54c97664d7fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.993624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf05684fcd942f99a52562ecada3cb3', 'request_id': 'req-6afd76bd-3754-4d48-b29e-54c97664d7fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:26.999690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf05684fcd942f99a52562ecada3cb3', 'request_id': 'req-6afd76bd-3754-4d48-b29e-54c97664d7fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:27.014864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] The token cache is being invalidate because project bbf05684fcd942f99a52562ecada3cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:27.015195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:27.015867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6afd76bd-3754-4d48-b29e-54c97664d7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:27.017495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 913/1848] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:26 2022] DELETE /identity/v3/projects/bbf05684fcd942f99a52562ecada3cb3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:27.023235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.023704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.023823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.047152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:27.048371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:27.107153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 936/1849] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:27.116005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.116295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.118674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.136601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:27.137854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:27.146757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 914/1850] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:27.150914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.160638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48193882-ffb5-4c1b-9e7e-6a0d86712c86 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.165247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.165845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.166155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.166431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.167193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:27.167498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.169081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.181206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 937/1851] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:27.185295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.195381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7ec06-e290-4d23-8a3f-4c4ca16965d6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.200064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.200710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.200992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.201265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.201905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.203835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.340324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 915/1852] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:27.344464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.354606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b19d-2e66-4417-afd8-ef85eadceb72 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.422794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.428244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.428244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.428244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.428244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.428244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.557081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 938/1853] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:27.561136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.571073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025bd9e-b9d2-411a-95e8-87af9392c327 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.575504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.576085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.576359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.576626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.577285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.578900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.588625 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 916/1854] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:27.592247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.602697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc8606-1b67-441b-9cd0-02f7feb8deb2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.607292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.607889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.608171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.608443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.609096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.610723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad0251-c26c-4a25-aa8f-634af1098ec1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.625035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 939/1855] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:27.626239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.636922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a91f1-5588-4e55-a9e9-9d09fb6a1f35 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.641500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.642202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.642478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.642746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] PATH_INFO: `/v3/projects/721e26e996d64058a401797938e79619/users/dccea21e61574847aa37c3bdfee1684b/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.656981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=721e26e996d64058a401797938e79619, user_id=dccea21e61574847aa37c3bdfee1684b, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.659656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.739367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 917/1856] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:27 2022] PUT /identity/v3/projects/721e26e996d64058a401797938e79619/users/dccea21e61574847aa37c3bdfee1684b/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:27.744326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:27.750383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.750672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.750976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.780653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] MFA Rules not processed for user `dccea21e61574847aa37c3bdfee1684b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:27.781902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:27.850417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 940/1857] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:27.854843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:27.923884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc0e6f-6c2d-47fe-a90d-b46b1efdd893 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:27.939991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:27.940540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:27.940820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:27.941698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:27.942370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:27.944382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:27.999277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:28.041245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 918/1858] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:27 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:28.045237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.056197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be8f3d-88af-49db-bf4c-b2fe1761055c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.117668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.118327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.118609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.118882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.119559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.121461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.238241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 941/1859] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:28 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:28.242904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.253024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b3b9-6a46-442a-a537-3ca43e41b5bf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.258413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.259011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.259291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.259565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.260252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.261852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.272520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 919/1860] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:28.276518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.286899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b61d2-d4e9-46d3-ae44-37884334f72e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.291451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.292084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.292358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.292631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] PATH_INFO: `/v3/projects/b50946ab7bb44df8a78867619baffaa3/users/8985c3123a72437097233004162bac57/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.308133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50946ab7bb44df8a78867619baffaa3, user_id=8985c3123a72437097233004162bac57, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.310874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.441821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 942/1861] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:28 2022] PUT /identity/v3/projects/b50946ab7bb44df8a78867619baffaa3/users/8985c3123a72437097233004162bac57/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:28.446026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.460827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340ffd5-fb68-4aa5-ac13-dbc0c6ed930f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.518003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.518649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.518951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.519204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.519917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.521884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34133a7-8fef-4f42-9586-87928bfebfec admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.531927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 920/1862] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:46:28.536587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.547649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc022-4b21-4ece-88be-f3c82c217a2c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.553423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.554182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.554461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.554731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] PATH_INFO: `/v3/projects/b50946ab7bb44df8a78867619baffaa3/users/8985c3123a72437097233004162bac57/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.556644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50946ab7bb44df8a78867619baffaa3, user_id=8985c3123a72437097233004162bac57, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.559295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.570536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 943/1863] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:28 2022] PUT /identity/v3/projects/b50946ab7bb44df8a78867619baffaa3/users/8985c3123a72437097233004162bac57/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:28.581134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.581448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.581721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.609489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] MFA Rules not processed for user `8985c3123a72437097233004162bac57`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:28.610720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:28.680345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 921/1864] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:28.691670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.693593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:28.756266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeb940-f982-4616-868d-462fcd103e2e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.759958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.760563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.760842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.761124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:28.770000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:28.772013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:28.781282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 944/1865] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:28.931782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:28.941180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716fead-23bf-4076-9e31-d954ecada3d1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:28.944830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:28.956322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:28.956322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:28.956322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:29.003038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:29.004610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e96c85-0d22-464d-bc27-b19951315159 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:29.014272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 922/1866] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:29.271152 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:29.608538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:29.683385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bc599-0352-444e-a5ca-ce40ae4ed9e8 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:29.688049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:29.688752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:29.689160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:29.689447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] PATH_INFO: `/v3/users/dccea21e61574847aa37c3bdfee1684b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:29.690616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccea21e61574847aa37c3bdfee1684b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:29.693058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:29.718840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:29.864090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccea21e61574847aa37c3bdfee1684b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:29.876672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccea21e61574847aa37c3bdfee1684b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:29.883384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccea21e61574847aa37c3bdfee1684b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:29.884062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccea21e61574847aa37c3bdfee1684b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:29.949399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccea21e61574847aa37c3bdfee1684b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:29.955541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 945/1867] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:29 2022] DELETE /identity/v3/users/dccea21e61574847aa37c3bdfee1684b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:29.959999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:29.961088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:29.961396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:29.981705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:29.982989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:30.042525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 923/1868] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:30.053354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:30.054832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:30.109985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486d600-5b89-4bcb-95d4-3550c880e27f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:30.113590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:30.116173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:30.116463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:30.116732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:30.125802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:30.127303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:30.137453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 946/1869] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:30.739648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:30.739648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:30.760281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff9707-8eef-4467-96d2-119171b2266d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:30.777866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:30.777866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:30.778621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:30.778621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] PATH_INFO: `/v3/projects/721e26e996d64058a401797938e79619` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:30.780213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721e26e996d64058a401797938e79619)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:30.783404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:30.796859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:30.798458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:30.812639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7936f-423e-4631-b8d3-2642590aa4cb service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:30.817993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:30.818670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:30.818947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:30.819216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] PATH_INFO: `/v3/users/99d4adc8e7e2489ba05f7425104f9281` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:30.820340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d4adc8e7e2489ba05f7425104f9281)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:30.822275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:30.914303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721e26e996d64058a401797938e79619', 'request_id': 'req-6a7a6140-ed71-4e24-9376-15eb2da25ca2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:30.987692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:31.018307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721e26e996d64058a401797938e79619', 'request_id': 'req-6a7a6140-ed71-4e24-9376-15eb2da25ca2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.025315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721e26e996d64058a401797938e79619', 'request_id': 'req-6a7a6140-ed71-4e24-9376-15eb2da25ca2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.033197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d4adc8e7e2489ba05f7425104f9281'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.037494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] The token cache is being invalidate because project 721e26e996d64058a401797938e79619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:31.037963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.038652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.040174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 924/1870] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:30 2022] DELETE /identity/v3/projects/721e26e996d64058a401797938e79619 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:31.044432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:31.045863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:31.045983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d4adc8e7e2489ba05f7425104f9281'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.053565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d4adc8e7e2489ba05f7425104f9281'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.054318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d4adc8e7e2489ba05f7425104f9281'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.108914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a6140-ed71-4e24-9376-15eb2da25ca2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:31.113527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:31.114689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:31.114969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:31.115241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] PATH_INFO: `/v3/users/8985c3123a72437097233004162bac57` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:31.116393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8985c3123a72437097233004162bac57)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:31.118357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:31.184115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d66ce6e-60bf-4f0f-b7dc-b7c775a5276a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d4adc8e7e2489ba05f7425104f9281'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.190526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 947/1871] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:30 2022] DELETE /identity/v3/users/99d4adc8e7e2489ba05f7425104f9281 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:31.195432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:31.195713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:31.195987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:31.214539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:31.215741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:31.283188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 948/1872] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:31.304266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:31.305970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:31.387708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8985c3123a72437097233004162bac57'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.413671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8985c3123a72437097233004162bac57'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.425810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:31.429699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8985c3123a72437097233004162bac57'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.436070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb60e-e0d1-440f-b57a-5563fa9c222b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:31.444556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:31.445415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:31.445766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:31.446133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:31.466970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:31.469220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:31.483120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 949/1873] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:31.652857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8985c3123a72437097233004162bac57'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.653527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8985c3123a72437097233004162bac57'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:31.661761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 925/1874] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:31 2022] DELETE /identity/v3/users/8985c3123a72437097233004162bac57 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:32.530696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:32.533619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:32.548574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c03ee8-3ec6-453f-b60f-915c54355464 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:32.666266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:32.667086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:32.667334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:32.667681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] PATH_INFO: `/v3/projects/b50946ab7bb44df8a78867619baffaa3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:32.669704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50946ab7bb44df8a78867619baffaa3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:32.672514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:32.756293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:32.769888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:32.775856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:32.794823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2c750-0de3-4e36-873b-6306b486ccf8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:32.800771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:32.801849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:32.802327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:32.802948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] PATH_INFO: `/v3/projects/3cf05fb1efd943c89fef84b661fa80d9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:32.818529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf05fb1efd943c89fef84b661fa80d9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:32.820526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:32.868058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50946ab7bb44df8a78867619baffaa3', 'request_id': 'req-d9f60673-7ed2-49ad-87f3-fde93a5ae346'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:32.891707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50946ab7bb44df8a78867619baffaa3', 'request_id': 'req-d9f60673-7ed2-49ad-87f3-fde93a5ae346'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:32.904975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50946ab7bb44df8a78867619baffaa3', 'request_id': 'req-d9f60673-7ed2-49ad-87f3-fde93a5ae346'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:32.999919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf05fb1efd943c89fef84b661fa80d9', 'request_id': 'req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.030163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] The token cache is being invalidate because project b50946ab7bb44df8a78867619baffaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:33.033344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.033344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d9f60673-7ed2-49ad-87f3-fde93a5ae346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.033344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 950/1875] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:32 2022] DELETE /identity/v3/projects/b50946ab7bb44df8a78867619baffaa3 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:33.037974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e39c3-4b66-4bc7-ade8-1e5318c58c60 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.038767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e39c3-4b66-4bc7-ade8-1e5318c58c60 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.039313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e39c3-4b66-4bc7-ade8-1e5318c58c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.066202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1b2e39c3-4b66-4bc7-ade8-1e5318c58c60 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:33.068108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1b2e39c3-4b66-4bc7-ade8-1e5318c58c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:33.113673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf05fb1efd943c89fef84b661fa80d9', 'request_id': 'req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.125547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf05fb1efd943c89fef84b661fa80d9', 'request_id': 'req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.146596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] The token cache is being invalidate because project 3cf05fb1efd943c89fef84b661fa80d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:33.147188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.148289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.150155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 926/1876] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:32 2022] DELETE /identity/v3/projects/3cf05fb1efd943c89fef84b661fa80d9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:33.154902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.165416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:33.184446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 951/1877] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:33.193532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.193532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.199040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.218509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:33.219881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:33.229800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 952/1878] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:33.236958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.251334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdffe4f-5389-45c4-a2ec-49c247a62bc1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.251848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f02b9b-3770-41e0-a180-344aa0afd75e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.258774 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.263046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.263046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.263046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] PATH_INFO: `/v3/users/775d82b6b32c486bb623aa3defb853b5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.263046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=775d82b6b32c486bb623aa3defb853b5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.263586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.264387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.265544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.265930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.266341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.268087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:33.268520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.270605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.288757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 953/1879] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:33.289775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.300052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac77a8-ff80-4964-97b0-95a696752d75 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.305853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.306500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.306815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.307086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.307721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.309692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.486104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 954/1880] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:33 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:33.490847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.569613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a62b7-9ac5-49c3-b174-a174d2f7aa67 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.574579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.575180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.575463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.575738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.576426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.578529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.601045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775d82b6b32c486bb623aa3defb853b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.615289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775d82b6b32c486bb623aa3defb853b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.625402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775d82b6b32c486bb623aa3defb853b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.691744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 955/1881] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:33 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:33.695252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.705562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0eac5-724e-4359-98c3-ae1b272901f5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.709607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775d82b6b32c486bb623aa3defb853b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.710390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775d82b6b32c486bb623aa3defb853b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:33.711127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.711834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.712248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.712530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.713262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.715068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.718237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 927/1882] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:33 2022] DELETE /identity/v3/users/775d82b6b32c486bb623aa3defb853b5 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:33.727331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 956/1883] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:33.731507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.733054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:33.823742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085250cd-06ad-4105-9eb5-930e1ba5df33 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.829125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.829758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.830040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.830356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.831217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.833205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b570ae-9419-4a46-b41b-9eb82f5e9a8f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:33.850715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 928/1884] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:33.851726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:33.865157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b392702-704c-4b38-a40c-a9c55290dd8d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:33.870645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:33.871323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:33.871601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:33.871876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] PATH_INFO: `/v3/projects/ff3d98c0db8e43d5adb495de652eef9e/users/0a207960510546749a1de7ae1e99feb3/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:33.889260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3d98c0db8e43d5adb495de652eef9e, user_id=0a207960510546749a1de7ae1e99feb3, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:33.893222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:34.001556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:34.018530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 957/1885] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:33 2022] PUT /identity/v3/projects/ff3d98c0db8e43d5adb495de652eef9e/users/0a207960510546749a1de7ae1e99feb3/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:34.029464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9126c-f029-477d-8471-e575067ade54 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:34.029772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9126c-f029-477d-8471-e575067ade54 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:34.030045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9126c-f029-477d-8471-e575067ade54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:34.085318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-09b9126c-f029-477d-8471-e575067ade54 None None] MFA Rules not processed for user `0a207960510546749a1de7ae1e99feb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:34.087434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-09b9126c-f029-477d-8471-e575067ade54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:34.185693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 929/1886] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:34.233114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:34.235014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:34.314193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2a26b-d55d-450c-b997-ff6b523cc701 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:34.318932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:34.320090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:34.321941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:34.322517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:34.333658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:34.335261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:34.346815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 958/1887] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:34.365098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9126c-f029-477d-8471-e575067ade54 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:34.458830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9126c-f029-477d-8471-e575067ade54 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:34.468676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:34.469529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:34.470166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:34.470524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] PATH_INFO: `/v3/users/0a207960510546749a1de7ae1e99feb3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:34.471791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a207960510546749a1de7ae1e99feb3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:34.474337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:34.562326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:34.719123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a207960510546749a1de7ae1e99feb3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:34.747219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a207960510546749a1de7ae1e99feb3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:34.755508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a207960510546749a1de7ae1e99feb3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:34.804220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:34.806170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:34.822185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdecae-9b76-462d-adc0-419d0d691bde service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:34.826832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:34.827464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:34.827770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:34.831051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] PATH_INFO: `/v3/projects/cefcb6e7201346e393b7cb616d6a15d8` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:34.831051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cefcb6e7201346e393b7cb616d6a15d8)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:34.831816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:34.886865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a207960510546749a1de7ae1e99feb3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:34.888133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-566b62b3-8001-49d0-a5d5-5310df70f99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a207960510546749a1de7ae1e99feb3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:34.898433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 930/1888] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:34 2022] DELETE /identity/v3/users/0a207960510546749a1de7ae1e99feb3 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:34.902994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:34.903594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:34.904076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:34.933448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:34.935646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.009132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cefcb6e7201346e393b7cb616d6a15d8', 'request_id': 'req-e4ab201a-69a1-4a91-9971-0520d92a2fa4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.035473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cefcb6e7201346e393b7cb616d6a15d8', 'request_id': 'req-e4ab201a-69a1-4a91-9971-0520d92a2fa4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.048988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cefcb6e7201346e393b7cb616d6a15d8', 'request_id': 'req-e4ab201a-69a1-4a91-9971-0520d92a2fa4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.059338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 931/1889] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:35.060341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:35.067963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.084385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba071f8-46ce-4a7d-be33-31d94c53143e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:35.124330 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] The token cache is being invalidate because project cefcb6e7201346e393b7cb616d6a15d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:35.124620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.125253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.126897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 959/1890] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:34 2022] DELETE /identity/v3/projects/cefcb6e7201346e393b7cb616d6a15d8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:35.127900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:35.129199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.144407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:35.145201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:35.145482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:35.145756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] PATH_INFO: `/v3/users/0fd3fa56688b4e999619ab6c4480faa3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:35.146934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd3fa56688b4e999619ab6c4480faa3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:35.148973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:35.195177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab201a-69a1-4a91-9971-0520d92a2fa4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:35.198744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:35.199358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:35.199635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:35.199906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:35.201632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.213633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:35.215080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:35.225006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 960/1891] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:35.225799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:35.227082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.238385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ad1d5-637e-4896-ac24-b5dbceaa594c service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:35.244202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:35.244820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:35.245096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:35.245362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] PATH_INFO: `/v3/users/b7123c0813b34f3c9b467630f6884ea5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:35.247285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7123c0813b34f3c9b467630f6884ea5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:35.249463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:35.481206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd3fa56688b4e999619ab6c4480faa3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.526751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd3fa56688b4e999619ab6c4480faa3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.544578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7123c0813b34f3c9b467630f6884ea5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.545934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd3fa56688b4e999619ab6c4480faa3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.561553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7123c0813b34f3c9b467630f6884ea5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.569797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7123c0813b34f3c9b467630f6884ea5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.570780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7123c0813b34f3c9b467630f6884ea5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.695702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd3fa56688b4e999619ab6c4480faa3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.696332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-daf785a6-25ef-41a0-a23a-23ebcd788a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd3fa56688b4e999619ab6c4480faa3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.703003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 932/1892] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:35 2022] DELETE /identity/v3/users/0fd3fa56688b4e999619ab6c4480faa3 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:35.707926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:35.708246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:35.708518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:35.727925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:35.731759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.795412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7123c0813b34f3c9b467630f6884ea5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:35.803311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 961/1893] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:35 2022] DELETE /identity/v3/users/b7123c0813b34f3c9b467630f6884ea5 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:35.806682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 933/1894] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:35.810438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:35.815137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.853033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:35.854588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:35.916247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c8ccf-1683-4855-8f2a-bf51454ae849 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:35.932160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:35.933841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:35.934686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:35.935576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] PATH_INFO: `/v3/projects/ff3d98c0db8e43d5adb495de652eef9e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:35.937004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3d98c0db8e43d5adb495de652eef9e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:35.939809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:35.959318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a040226-c03d-4701-b96f-2321822bcd02 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.030072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.030740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.031153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.031425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.033229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.045736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.047213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f879dc8-11ca-472a-96bb-2135fd6e9fe9 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.063691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 934/1895] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:36.104482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3d98c0db8e43d5adb495de652eef9e', 'request_id': 'req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.111512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3d98c0db8e43d5adb495de652eef9e', 'request_id': 'req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.121446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3d98c0db8e43d5adb495de652eef9e', 'request_id': 'req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.297517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] The token cache is being invalidate because project ff3d98c0db8e43d5adb495de652eef9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:36.297836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.298495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7db536e4-da7b-49bf-8bb5-c14c73ae1d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.300231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 962/1896] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:35 2022] DELETE /identity/v3/projects/ff3d98c0db8e43d5adb495de652eef9e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:36.305310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.305601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.305883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.325766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:36.327035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.398313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 935/1897] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:36.407110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.407473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.407817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.429114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:36.430445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.438912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 963/1898] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:36.443276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:36.453032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed99bd-a1d9-4f31-8cc5-3fcb19d06a2d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.458388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.458994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.459959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.459959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.460363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:36.460672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.462311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.474484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 936/1899] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:36.478938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:36.488212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350eeed-76ea-449b-a202-e8f1fadc16b8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.492802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:36.494766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.495415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.496104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.496390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.496668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.497327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.499546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.512791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db186461-1bfe-41cf-b13e-70096415041b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.516081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:36.520848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.521551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.521822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.522126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] PATH_INFO: `/v3/projects/c3f58efab891462bab20dc053b873910` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.523278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f58efab891462bab20dc053b873910)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.525238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.619758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 964/1900] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:36 2022] POST /identity/v3/projects => generated 363 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:36.619758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:36.620463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.690430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:36.692765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f58efab891462bab20dc053b873910', 'request_id': 'req-ec607665-50d3-4071-8749-bba5861c870c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.734772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9438c-7f44-4a92-b9cd-497bcc05c519 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.781354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f58efab891462bab20dc053b873910', 'request_id': 'req-ec607665-50d3-4071-8749-bba5861c870c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.788115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f58efab891462bab20dc053b873910', 'request_id': 'req-ec607665-50d3-4071-8749-bba5861c870c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.822754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] The token cache is being invalidate because project c3f58efab891462bab20dc053b873910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:36.823293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.824162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:36.826452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 937/1901] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:36 2022] DELETE /identity/v3/projects/c3f58efab891462bab20dc053b873910 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:36.827385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:36.828976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:36.830228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.831535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.831822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.832097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] PATH_INFO: `/v3/projects/53883b89cb56420faf08f6c002710255` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.844651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53883b89cb56420faf08f6c002710255)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.848231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec607665-50d3-4071-8749-bba5861c870c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:36.848349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.855689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:36.856442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.856943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.857753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:36.858966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:36.861009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c74e3-f0ec-408b-90fa-7a8ca3268197 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:36.998806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 938/1902] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:36 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:36.998806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:36.998806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:36.998806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.022310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53883b89cb56420faf08f6c002710255', 'request_id': 'req-bd3d2008-cabc-4fed-8f7e-d745479d9c81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.024472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:37.026894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.031674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53883b89cb56420faf08f6c002710255', 'request_id': 'req-bd3d2008-cabc-4fed-8f7e-d745479d9c81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.039625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53883b89cb56420faf08f6c002710255', 'request_id': 'req-bd3d2008-cabc-4fed-8f7e-d745479d9c81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.095362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 939/1903] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:37.097254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.108777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935d68-3278-44c7-a2f5-5098ba6d461e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.114241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.114835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.115114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.115386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.116036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.117663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273adbd-312e-4b3a-bbf9-4580495a2708 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.130527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 940/1904] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:37.131840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.132124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.132392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.153553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:37.154833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.167701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 941/1905] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:37.167701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.173284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] The token cache is being invalidate because project 53883b89cb56420faf08f6c002710255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:37.173766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.174632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.176278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37fcf6-6f04-4104-ab9a-0d30b447adb4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.176625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 965/1906] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:36 2022] DELETE /identity/v3/projects/53883b89cb56420faf08f6c002710255 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:37.178100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.179888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.182003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.182842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.183289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.183706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.184629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.186637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.201067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d2008-cabc-4fed-8f7e-d745479d9c81 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.211533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.212365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.212778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.213185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.214654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:37.215106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.217225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.227304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 942/1907] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:37.228887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.235357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.251254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 966/1908] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:37.255445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.257590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.273559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c735-fee4-4d01-8fbd-2b002ed4dbd0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.280032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.280032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.280292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.280635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] PATH_INFO: `/v3/projects/c27ebd8558f641a785c7da8d4751dfb8/users/4ca49497e7cd43f48ff437eabdeea708/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.282014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734a46c-3a24-4b14-b9a8-7dff2ce68fdb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.288425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.289502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.290007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.290480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] PATH_INFO: `/v3/users/d03636ad79374ea388da66f9a5a5edd2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.291938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03636ad79374ea388da66f9a5a5edd2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.294166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.299952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27ebd8558f641a785c7da8d4751dfb8, user_id=4ca49497e7cd43f48ff437eabdeea708, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.303148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.452490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 967/1909] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:37 2022] PUT /identity/v3/projects/c27ebd8558f641a785c7da8d4751dfb8/users/4ca49497e7cd43f48ff437eabdeea708/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:37.453310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.522941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec656bee-e6fa-4227-bbac-5eaac03fcbd4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.527702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.528318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.528601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.528876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.529518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.535966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397ea2f-ffca-46bb-8906-a0284976cf87 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.549018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03636ad79374ea388da66f9a5a5edd2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.563139 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03636ad79374ea388da66f9a5a5edd2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.569234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03636ad79374ea388da66f9a5a5edd2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.665122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03636ad79374ea388da66f9a5a5edd2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.665735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03636ad79374ea388da66f9a5a5edd2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:37.671826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 968/1910] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:37 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:37.672908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.675862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.675862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.676597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 943/1911] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:37 2022] DELETE /identity/v3/users/d03636ad79374ea388da66f9a5a5edd2 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:37.677114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.678846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.712362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] MFA Rules not processed for user `4ca49497e7cd43f48ff437eabdeea708`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:37.713602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.760485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c1f3e-afd8-4e29-a4ae-f823e1e80f18 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.765968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.767044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.767275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.767552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.768194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.770225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.782748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 969/1912] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:37.794572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.795988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:37.861443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239833bd-4f48-4f28-abe4-da4fb60ccc7a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.865677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.866301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.866578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.866846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.875058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.876469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.888244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 944/1913] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:37 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:37.888420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 970/1914] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:37.896095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.909543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2260931-0fe7-425d-ab9e-7b788d34799f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.914877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.916985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.916985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.916985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.916985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.918175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.944255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 945/1915] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:37.960524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:37.972595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d232-0abb-428d-9d79-182904031a4a service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:37.979097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:37.980077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:37.985786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:37.985786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:37.985786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:37.985786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c0a88-afa6-4e4c-bca7-9c530bcd144c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:37.998349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 971/1916] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:38.001934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.013958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df266d-09ce-4bd3-9ec3-606ff30d433b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:38.019586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:38.020282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.020561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.029367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] PATH_INFO: `/v3/projects/c538a572650b4574b9f6f9c8853e9a4f/users/60d10ce14ee3486f8e714dcc912d1527/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.040370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c538a572650b4574b9f6f9c8853e9a4f, user_id=60d10ce14ee3486f8e714dcc912d1527, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:38.043299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:38.130917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 946/1917] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:38 2022] PUT /identity/v3/projects/c538a572650b4574b9f6f9c8853e9a4f/users/60d10ce14ee3486f8e714dcc912d1527/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:38.136112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.136431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.136713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.156838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:38.158099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:38.220662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 972/1918] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:38.231234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.233103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:38.240992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.243110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:38.258041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16ae40-db23-449a-b9fc-a313fbbc3c32 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:38.265565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:38.266371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.266773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.267124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] PATH_INFO: `/v3/projects/997e303e088648a0a579dd05fafc2e1c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.269013 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] RBAC: Authorizing `identity:delete_project(project_id=997e303e088648a0a579dd05fafc2e1c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:38.276783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:38.277242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:38.307054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d956-11f2-4ec4-b13a-0e0c85a51e66 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:38.311771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:38.312501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.312770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.313044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.323113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:38.324553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f128e-2c6a-4c2e-9233-13a25f7fd119 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:38.334672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 947/1919] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:38.448834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997e303e088648a0a579dd05fafc2e1c', 'request_id': 'req-c2705d9b-89dd-4702-97a7-e42cdf35304b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:38.456075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997e303e088648a0a579dd05fafc2e1c', 'request_id': 'req-c2705d9b-89dd-4702-97a7-e42cdf35304b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:38.483000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997e303e088648a0a579dd05fafc2e1c', 'request_id': 'req-c2705d9b-89dd-4702-97a7-e42cdf35304b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:38.614627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] The token cache is being invalidate because project 997e303e088648a0a579dd05fafc2e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:38.614924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:38.615835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2705d9b-89dd-4702-97a7-e42cdf35304b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:38.618124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 973/1920] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:38 2022] DELETE /identity/v3/projects/997e303e088648a0a579dd05fafc2e1c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:38.623097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.623520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.624079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.647063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:38.648499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:38.724383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 948/1921] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:38.733581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.733970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.734297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.807488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:38.809339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:38.818544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 974/1922] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:38.822843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.833872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2f03-d191-444e-99d8-fe8c1f054496 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:38.839314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:38.840007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.840316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.840681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.841568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:38.841817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:38.843565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:38.858687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 949/1923] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:38.862429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.873603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dc4eb-54da-4c5c-b53f-ee43e45f7188 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:38.886515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:38.887119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:38.887464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:38.887689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:38.892705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:38.892705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:38.970994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:38.978616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 975/1924] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:38 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:38.982936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:38.997502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ef85-825c-4f8e-bc20-986ef6ea03c3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.076630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.077392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.077746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.078115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.078819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.080820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:39.200904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:39.210863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 950/1925] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:38 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:39.211798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:39.222458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1a314-a1b5-4a91-884e-1854244674d8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.227708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.228327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.228674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.228973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.229624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.231245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:39.242088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 976/1926] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:46:39.246120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:39.258029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4022c-8b69-4b17-acda-0753a7dcc2cf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.262679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.263317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.263600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.264080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.264737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.266438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72791-e37d-4b22-8f0a-36da5bdfe531 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:39.277850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 951/1927] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:46:39.282153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:39.293153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd07f5-6a32-4ffd-9d57-d1f0f2cea08e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.297880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.298597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.298880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.299175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] PATH_INFO: `/v3/projects/14aaf111e2744716ae5573a1c5ea8ddb/users/14db1fa6e791403e905306a4d145def2/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.314954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aaf111e2744716ae5573a1c5ea8ddb, user_id=14db1fa6e791403e905306a4d145def2, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.317778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:39.417536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 977/1928] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:39 2022] PUT /identity/v3/projects/14aaf111e2744716ae5573a1c5ea8ddb/users/14db1fa6e791403e905306a4d145def2/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:39.429414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.430236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.430516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.466692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] MFA Rules not processed for user `14db1fa6e791403e905306a4d145def2`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:39.468269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:39.538880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 952/1929] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:39.545011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:39.642813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42f68-8626-4656-aeea-57e4b2a61520 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.656253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.656253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.656253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.656253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.657066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.659533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:39.782713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:39.782902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 978/1930] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:39 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:39.799574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08151-431e-4797-9f23-7b69cdd41ebf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:39.878776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:39.879641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:39.879957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:39.880309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:39.880985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:39.895566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.038531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 953/1931] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:39 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:40.042220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.054535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c094c-c018-4ad4-b947-6e09b74319a9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.060582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.061376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.061950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.062407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.063397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.065313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.083235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 979/1932] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:40.084807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.094988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db0176-72bf-49a9-a102-d288bc82192c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.099991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.100619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.100902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.101174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.101887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.103509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e083-c34b-450e-8c9f-4a49d6d4c6d9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.128297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 954/1933] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:40.130149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.145806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0596c9-841d-43f5-9b91-9fe9512841b3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.150676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.151350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.151635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.151914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] PATH_INFO: `/v3/projects/60348436798e40428831bea52533f6f5/users/21e292930b684502a8fd1dae793b196e/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.168834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=60348436798e40428831bea52533f6f5, user_id=21e292930b684502a8fd1dae793b196e, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.171541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.282454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:40.299198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 980/1934] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:40 2022] PUT /identity/v3/projects/60348436798e40428831bea52533f6f5/users/21e292930b684502a8fd1dae793b196e/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:40.301068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d90c4-fd24-4472-9851-efa20dccd380 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.301626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d90c4-fd24-4472-9851-efa20dccd380 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.302592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d90c4-fd24-4472-9851-efa20dccd380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.358722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-951d90c4-fd24-4472-9851-efa20dccd380 None None] MFA Rules not processed for user `21e292930b684502a8fd1dae793b196e`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:40.360941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-951d90c4-fd24-4472-9851-efa20dccd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:40.475820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 955/1935] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:40.480989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.538129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.538692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.539208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.605525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] MFA Rules not processed for user `60d10ce14ee3486f8e714dcc912d1527`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:40.607377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:40.654846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0203e4-3b47-46f6-926d-742c63edfccd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.662351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.663304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.663796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.664305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.665231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.668522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.703351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 956/1936] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:40.715291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.727007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:40.740766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:40.779511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 981/1937] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:40 2022] POST /identity/v3/projects => generated 379 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:40.781764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:40.813558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbdb39-f79a-4872-8ca8-411350f7275e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.863834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ad8b-b3f6-41c3-8b52-d4363aa275d6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:40.869262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.869780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.870363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.870521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.871230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.873250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.880190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:40.880872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:40.883441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:40.883441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:40.940155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:40.941684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:40.951704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 957/1938] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:40.988548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 982/1939] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:40 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:40.989583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:41.004269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2aaf3-7ea0-4766-8e00-cf8860c177bf service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:41.010566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:41.011195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.011786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.012070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.012728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:41.014368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:41.028852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 958/1940] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:41.032919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:41.047715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8cd7-ace4-4b43-9453-7e4a93499275 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:41.055811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:41.070754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.070754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.070754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] PATH_INFO: `/v3/projects/474867eda69a450dbb2f8de01fe260d9/users/aa40bd3961534db8ac6335b964e6b468/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.077855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474867eda69a450dbb2f8de01fe260d9, user_id=aa40bd3961534db8ac6335b964e6b468, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:41.081019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:41.163973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 983/1941] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:41 2022] PUT /identity/v3/projects/474867eda69a450dbb2f8de01fe260d9/users/aa40bd3961534db8ac6335b964e6b468/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:41.169099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:41.186142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24ae4-a24a-4cbf-9e73-da9c0fc628a3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:41.265248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:41.266125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.266647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.266931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.267633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:41.269386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbd2ad-42a8-45b3-9fe2-57b0463d3407 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:41.287481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 959/1942] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:41.295688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:41.316258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237763-671c-4ffd-9229-e039509ba0d8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:41.322191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:41.322957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.323938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.324327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] PATH_INFO: `/v3/projects/474867eda69a450dbb2f8de01fe260d9/users/aa40bd3961534db8ac6335b964e6b468/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.326599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474867eda69a450dbb2f8de01fe260d9, user_id=aa40bd3961534db8ac6335b964e6b468, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:41.329740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:41.345456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 984/1943] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:41 2022] PUT /identity/v3/projects/474867eda69a450dbb2f8de01fe260d9/users/aa40bd3961534db8ac6335b964e6b468/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:41.358243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.358243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.358888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.406465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] MFA Rules not processed for user `aa40bd3961534db8ac6335b964e6b468`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:41.407808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:41.504028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:41.608551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 960/1944] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:41.834877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:41.915539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402508d-1068-42cf-b091-8233bd359c50 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:41.920461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:41.923588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:41.923588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:41.923588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:41.931216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:41.932738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0e087-b7fd-46b2-b7e6-1efa6d98fb7a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:41.942965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 985/1945] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:42.127359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:42.186046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:42.196323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7616c-c4c6-4971-ae68-3b9f92f13e18 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:42.200337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:42.200946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:42.207382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:42.207382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:42.264141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:42.265731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:42.275855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 961/1946] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:43.295782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a95b-8c7c-44d4-bbf3-d95f67b6020c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:43.296263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a95b-8c7c-44d4-bbf3-d95f67b6020c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:43.296357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823a95b-8c7c-44d4-bbf3-d95f67b6020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:43.331242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e823a95b-8c7c-44d4-bbf3-d95f67b6020c None None] MFA Rules not processed for user `60d10ce14ee3486f8e714dcc912d1527`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:43.334786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e823a95b-8c7c-44d4-bbf3-d95f67b6020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:43.420412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 986/1947] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:44.127245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:44.389940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:47.724421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:48.158857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:48.322911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:48.414604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3b07-66a4-4237-97ab-50d9c4843cc8 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:48.419128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:48.419762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:48.420123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:48.420401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] PATH_INFO: `/v3/users/14db1fa6e791403e905306a4d145def2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:48.421555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14db1fa6e791403e905306a4d145def2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:48.423761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:48.702445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14db1fa6e791403e905306a4d145def2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:48.716063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14db1fa6e791403e905306a4d145def2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:48.725367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14db1fa6e791403e905306a4d145def2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:48.806213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14db1fa6e791403e905306a4d145def2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:48.806586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14db1fa6e791403e905306a4d145def2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:48.820504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 962/1948] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:48 2022] DELETE /identity/v3/users/14db1fa6e791403e905306a4d145def2 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:48.825658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:48.826474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:48.827223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:48.848945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:48.851434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:48.956160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 987/1949] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:46:48.970882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:48.974859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:49.079018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb73cb-7adc-427a-9de2-80af08e36476 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:49.083356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:49.086592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:49.087267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:49.087770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:49.108061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:49.109701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:49.128423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 963/1950] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:46:49.617379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:49.619633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:49.645551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69519b-fac8-4da6-a65c-d21177b397dc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:49.652090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:49.652738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:49.653515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:49.654250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] PATH_INFO: `/v3/projects/14aaf111e2744716ae5573a1c5ea8ddb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:49.656395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=14aaf111e2744716ae5573a1c5ea8ddb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:49.659113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:49.784699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14aaf111e2744716ae5573a1c5ea8ddb', 'request_id': 'req-6990f4a9-8c38-449b-a2dc-837cea3178fc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:49.791784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14aaf111e2744716ae5573a1c5ea8ddb', 'request_id': 'req-6990f4a9-8c38-449b-a2dc-837cea3178fc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:49.798293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14aaf111e2744716ae5573a1c5ea8ddb', 'request_id': 'req-6990f4a9-8c38-449b-a2dc-837cea3178fc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:49.855384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:49.891437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] The token cache is being invalidate because project 14aaf111e2744716ae5573a1c5ea8ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:49.893000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:49.893000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:49.894520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 988/1951] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:49 2022] DELETE /identity/v3/projects/14aaf111e2744716ae5573a1c5ea8ddb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:49.898284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:49.908053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aa2a4-d4be-4d15-b13e-3722f3fe9a1e service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:49.965839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:49.966715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:49.967285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:49.967439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] PATH_INFO: `/v3/users/21e292930b684502a8fd1dae793b196e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:49.968815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e292930b684502a8fd1dae793b196e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:49.972026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:50.314574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e292930b684502a8fd1dae793b196e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:50.327606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e292930b684502a8fd1dae793b196e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:50.335228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e292930b684502a8fd1dae793b196e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:50.427092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e292930b684502a8fd1dae793b196e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:50.428773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e292930b684502a8fd1dae793b196e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:50.434304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 964/1952] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:49 2022] DELETE /identity/v3/users/21e292930b684502a8fd1dae793b196e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:50.455288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:51.029195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:51.031545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:51.053711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f4a9-8c38-449b-a2dc-837cea3178fc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:51.134043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:51.135441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:51.135441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:51.135441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] PATH_INFO: `/v3/projects/60348436798e40428831bea52533f6f5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:51.136771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60348436798e40428831bea52533f6f5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:51.138770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:51.258484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60348436798e40428831bea52533f6f5', 'request_id': 'req-61caa173-87b1-443c-943e-131e37661520'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.264882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60348436798e40428831bea52533f6f5', 'request_id': 'req-61caa173-87b1-443c-943e-131e37661520'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.270610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60348436798e40428831bea52533f6f5', 'request_id': 'req-61caa173-87b1-443c-943e-131e37661520'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.367161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] The token cache is being invalidate because project 60348436798e40428831bea52533f6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:51.367710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.368662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.370256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 989/1953] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:51 2022] DELETE /identity/v3/projects/60348436798e40428831bea52533f6f5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:51.374463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:51.376505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:51.392252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979173f5-e3ab-446a-b3e5-6f0b6e590c44 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:51.452266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:51.452963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:51.453238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:51.453510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] PATH_INFO: `/v3/users/aa40bd3961534db8ac6335b964e6b468` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:51.454759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa40bd3961534db8ac6335b964e6b468)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:51.456895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:51.720694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa40bd3961534db8ac6335b964e6b468'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.739054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa40bd3961534db8ac6335b964e6b468'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.742992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa40bd3961534db8ac6335b964e6b468'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.814739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa40bd3961534db8ac6335b964e6b468'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.815691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-739e4fec-c598-44ae-977e-c3b77cb581c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa40bd3961534db8ac6335b964e6b468'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:51.822898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 965/1954] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:46:51 2022] DELETE /identity/v3/users/aa40bd3961534db8ac6335b964e6b468 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:46:52.251296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:52.330821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:52.336170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:52.345229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caa173-87b1-443c-943e-131e37661520 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:52.408892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:52.409579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:52.409898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:52.410697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] PATH_INFO: `/v3/projects/474867eda69a450dbb2f8de01fe260d9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:52.411860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] RBAC: Authorizing `identity:delete_project(project_id=474867eda69a450dbb2f8de01fe260d9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:52.415209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:52.552567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474867eda69a450dbb2f8de01fe260d9', 'request_id': 'req-906e89db-b96c-41b2-ba3f-205a381d498b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:52.558766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474867eda69a450dbb2f8de01fe260d9', 'request_id': 'req-906e89db-b96c-41b2-ba3f-205a381d498b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:52.564536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474867eda69a450dbb2f8de01fe260d9', 'request_id': 'req-906e89db-b96c-41b2-ba3f-205a381d498b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:52.577331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:52.667735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] The token cache is being invalidate because project 474867eda69a450dbb2f8de01fe260d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:46:52.677085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:52.677085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-906e89db-b96c-41b2-ba3f-205a381d498b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:46:52.677085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 990/1955] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:46:52 2022] DELETE /identity/v3/projects/474867eda69a450dbb2f8de01fe260d9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:52.682067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:52.682388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:52.682666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:52.702225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:52.703437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:52.760267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 966/1956] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:52.768478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:52.768761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:52.769036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:52.789620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:52.790851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:52.798704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 991/1957] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:52.803182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:52.812841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef832c-0e90-4cff-83f3-a3ced6459fa0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:52.817404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:52.818023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:52.818332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:52.818597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:52.819240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:46:52.819561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:52.821275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:52.833639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 967/1958] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:46:52.838040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:52.847367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eeada-c6b5-45c9-a457-a5d6a1cb81c1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:52.852616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:52.853218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:52.861355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:52.861355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:52.861355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:52.861355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.015212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 992/1959] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:52 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:46:53.015329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:53.015329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b201858-9749-456c-9445-71376d48ac66 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:53.069444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:53.070193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.070476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.070751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.071397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:53.073332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.198193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 968/1960] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:52 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:53.204610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:53.215822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77398c1-715f-428f-80f8-691eb011e4c9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:53.220760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:53.221019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.221293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.221589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.222285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:53.224064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.234735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 993/1961] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:53.239226 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:53.249412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fcae4-09cf-4634-ab25-e5e47aef2b21 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:53.254678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:53.255278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.255555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.255830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.256545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:53.258469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3dd64-12b4-4479-be8c-3638d5dd7648 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.269215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 969/1962] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:53.274157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:53.286828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce090-7d61-4fc6-8dad-561a1d0ee47a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:53.291890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:53.292626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.292983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.293334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] PATH_INFO: `/v3/projects/5b326c56b1354084acd9f78d2b0f53a9/users/60e6ccd7306840c68df78d95a9fa3d29/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.310529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b326c56b1354084acd9f78d2b0f53a9, user_id=60e6ccd7306840c68df78d95a9fa3d29, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:53.313266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.428892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:53.434425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 994/1963] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:53 2022] PUT /identity/v3/projects/5b326c56b1354084acd9f78d2b0f53a9/users/60e6ccd7306840c68df78d95a9fa3d29/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:53.439486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13a93-6648-42d2-b913-cf9c7b2e9688 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.439780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13a93-6648-42d2-b913-cf9c7b2e9688 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.440050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13a93-6648-42d2-b913-cf9c7b2e9688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.460840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-07d13a93-6648-42d2-b913-cf9c7b2e9688 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:53.462100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07d13a93-6648-42d2-b913-cf9c7b2e9688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:53.523766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 970/1964] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:53.533403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:53.535210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:53.576930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:53.590606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08937c-fbba-4936-85e5-283559c56b6a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:53.594221 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:53.594820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:53.595102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:53.595375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:53.604120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:53.605523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:53.615649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 995/1965] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:53.923278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:55.102761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:55.300185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:55.301410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:55.302169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:55.329678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] MFA Rules not processed for user `60e6ccd7306840c68df78d95a9fa3d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:55.330918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:55.406104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 971/1966] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:55.411386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:55.421442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b287c-770d-4c59-8fcf-36fe09a02c0c service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:55.427384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:55.428361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:55.428686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:55.429039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:55.429765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:55.431850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:55.626615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 996/1967] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:46:55 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:55.631123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:55.641449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb4b41-b93e-491f-b33e-215c3c1d85e5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:55.698652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:55.699316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:55.699600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:55.699873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:55.700501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:55.702459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:55.817989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 972/1968] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:46:55 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:46:55.822608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:55.833447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8e9f0-3bac-4e98-aa08-98bfeac3166a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:55.843094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:55.843732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:55.843999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:55.844274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:55.844926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:55.846654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:55.857216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 997/1969] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:55.861483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:55.872160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f87-a5d5-419d-a905-cfe0413f3e0c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:55.877217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:55.877967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:55.879456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:55.879456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] PATH_INFO: `/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c/users/21cbf2237b4246dbb8b0ea02fdd1cbdb/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:55.893934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6b32c2e0404fe7aeb7941e5f0abb7c, user_id=21cbf2237b4246dbb8b0ea02fdd1cbdb, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:55.897155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:56.009304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 973/1970] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:55 2022] PUT /identity/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c/users/21cbf2237b4246dbb8b0ea02fdd1cbdb/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:56.013285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:56.024010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2912e13-e7fe-46a0-ac3c-0cd56c9df1ff admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:56.080362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:56.080970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:56.081247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:56.081516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:56.082982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:56.084553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68134f9e-e58e-46f0-8432-2bd35f2c02f5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:56.094181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 998/1971] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:46:56.098586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:56.108950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb414aa-58ed-4b02-b526-195aa2b6859c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:56.113561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:56.114931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:56.114931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:56.114931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] PATH_INFO: `/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c/users/21cbf2237b4246dbb8b0ea02fdd1cbdb/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:56.116713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6b32c2e0404fe7aeb7941e5f0abb7c, user_id=21cbf2237b4246dbb8b0ea02fdd1cbdb, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:56.119370 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:56.130002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 974/1972] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:46:56 2022] PUT /identity/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c/users/21cbf2237b4246dbb8b0ea02fdd1cbdb/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:46:56.262832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:56.664985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:57.679655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:57.682415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:57.682415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:57.729265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] MFA Rules not processed for user `21cbf2237b4246dbb8b0ea02fdd1cbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:46:57.730680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:57.808430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 999/1973] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:46:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:46:57.827276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:46:57.829057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:46:57.886092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f22b7-15d8-40ce-b01b-45e3fc9707d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:46:57.889794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:46:57.890508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:46:57.890795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:46:57.891327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:46:57.950310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:46:57.951839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:46:57.961549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 975/1974] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:46:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:46:58.211392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:58.488996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:46:59.152006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:00.045639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:00.146089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:00.628878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:01.228177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:01.388080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:01.948583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.268057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.508254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.611011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.642878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.739540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:02.993998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:03.124643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:07.473928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:07.483520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa1f5c-1e25-40eb-92f5-6b047d5a08ba None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:07.487793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:07.488422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:07.488707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:07.488978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:07.498303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:07.499900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaad63e-4799-4de6-9286-edb035746bd6 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:07.509432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1000/1975] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:13.021688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:17.505182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:30.495974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:30.497510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:30.575849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba17a8a-ac86-4bee-92d8-be7970c02b54 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:30.581249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:30.581887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:30.582202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:30.582471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] PATH_INFO: `/v3/users/3d3a9ebe90a34b389d16eb3e01ca045c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:30.583604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3a9ebe90a34b389d16eb3e01ca045c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:30.585563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:30.816862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3a9ebe90a34b389d16eb3e01ca045c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:30.830124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3a9ebe90a34b389d16eb3e01ca045c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:30.840941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3a9ebe90a34b389d16eb3e01ca045c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:30.911262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3a9ebe90a34b389d16eb3e01ca045c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:30.914303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3a9ebe90a34b389d16eb3e01ca045c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:30.919048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 976/1976] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:30 2022] DELETE /identity/v3/users/3d3a9ebe90a34b389d16eb3e01ca045c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:30.923756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:30.926839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:30.927302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:30.958168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:30.959458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:31.029100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1001/1977] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:31.040465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:31.042506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:31.116746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993918b0-9891-498d-9ea0-99c866a4f92c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:31.120401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:31.121044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:31.121284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:31.121557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:31.133734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:31.135244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:31.167545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 977/1978] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:32.186008 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:32.187448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:32.205106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236802d-e324-40f6-883e-27041d13fd04 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:32.214905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:32.216863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:32.216863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:32.217043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] PATH_INFO: `/v3/projects/917d00134c234e7695460ce39ad9360a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:32.222743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=917d00134c234e7695460ce39ad9360a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:32.225044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:32.436512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917d00134c234e7695460ce39ad9360a', 'request_id': 'req-670b5d32-bd30-4a01-91de-a8a1381436ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.452150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917d00134c234e7695460ce39ad9360a', 'request_id': 'req-670b5d32-bd30-4a01-91de-a8a1381436ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.460166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917d00134c234e7695460ce39ad9360a', 'request_id': 'req-670b5d32-bd30-4a01-91de-a8a1381436ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.568346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] The token cache is being invalidate because project 917d00134c234e7695460ce39ad9360a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:32.568639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.569514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.571179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1002/1979] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:32 2022] DELETE /identity/v3/projects/917d00134c234e7695460ce39ad9360a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:32.575440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:32.586765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8307f-3a21-4298-abb2-c65f2c638203 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:32.609474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:32.652247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:32.652895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:32.653300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:32.653581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] PATH_INFO: `/v3/users/2f87dcabc72a4e66b489b5b8e2fc888d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:32.654753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f87dcabc72a4e66b489b5b8e2fc888d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:32.658788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:32.729504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:32.731180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:32.757081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b5d32-bd30-4a01-91de-a8a1381436ee admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:32.761911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:32.762581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:32.762856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:32.763125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] PATH_INFO: `/v3/users/60e6ccd7306840c68df78d95a9fa3d29` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:32.764393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e6ccd7306840c68df78d95a9fa3d29)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:32.766457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:32.870720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f87dcabc72a4e66b489b5b8e2fc888d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.886200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f87dcabc72a4e66b489b5b8e2fc888d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.893507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f87dcabc72a4e66b489b5b8e2fc888d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.952631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e6ccd7306840c68df78d95a9fa3d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.978909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e6ccd7306840c68df78d95a9fa3d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.988253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e6ccd7306840c68df78d95a9fa3d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.989030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e6ccd7306840c68df78d95a9fa3d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:32.993358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:33.003086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f87dcabc72a4e66b489b5b8e2fc888d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.003763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f87dcabc72a4e66b489b5b8e2fc888d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.018904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 978/1980] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:32 2022] DELETE /identity/v3/users/2f87dcabc72a4e66b489b5b8e2fc888d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:33.161017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e6ccd7306840c68df78d95a9fa3d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.170218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1003/1981] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:32 2022] DELETE /identity/v3/users/60e6ccd7306840c68df78d95a9fa3d29 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:33.514754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:33.515430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:33.638641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeadb3-28c5-4143-a691-57fd706cd6d2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:33.685668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:33.688309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:33.690706 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:33.724296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:33.725542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:33.725924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:33.726575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] PATH_INFO: `/v3/projects/5b326c56b1354084acd9f78d2b0f53a9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:33.728873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b326c56b1354084acd9f78d2b0f53a9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:33.732382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:33.771751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70626dfe-ac32-471f-b941-9b707fb96421 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:33.776694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:33.777309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:33.777609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:33.778046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] PATH_INFO: `/v3/projects/d7fac6f013d94c59801313e73ae2f3c0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:33.779318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fac6f013d94c59801313e73ae2f3c0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:33.781213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:33.858582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b326c56b1354084acd9f78d2b0f53a9', 'request_id': 'req-97c15cc2-a6ef-45e5-85da-ede2ca68829a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.920700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fac6f013d94c59801313e73ae2f3c0', 'request_id': 'req-8c79f2ba-e373-4e05-8035-e19c656747e9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.929029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b326c56b1354084acd9f78d2b0f53a9', 'request_id': 'req-97c15cc2-a6ef-45e5-85da-ede2ca68829a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.933713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fac6f013d94c59801313e73ae2f3c0', 'request_id': 'req-8c79f2ba-e373-4e05-8035-e19c656747e9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.936565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b326c56b1354084acd9f78d2b0f53a9', 'request_id': 'req-97c15cc2-a6ef-45e5-85da-ede2ca68829a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.941989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fac6f013d94c59801313e73ae2f3c0', 'request_id': 'req-8c79f2ba-e373-4e05-8035-e19c656747e9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.960989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] The token cache is being invalidate because project 5b326c56b1354084acd9f78d2b0f53a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:33.961283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.962014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:33.963533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 979/1982] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:33 2022] DELETE /identity/v3/projects/5b326c56b1354084acd9f78d2b0f53a9 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:33.967551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:33.968855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:34.036605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c15cc2-a6ef-45e5-85da-ede2ca68829a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.093853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.094522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.094803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.095078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] PATH_INFO: `/v3/users/21cbf2237b4246dbb8b0ea02fdd1cbdb` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.096729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] RBAC: Authorizing `identity:delete_user(user_id=21cbf2237b4246dbb8b0ea02fdd1cbdb)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.098830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.099949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] The token cache is being invalidate because project d7fac6f013d94c59801313e73ae2f3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:34.100302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.109662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8c79f2ba-e373-4e05-8035-e19c656747e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.109662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1004/1983] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:33 2022] DELETE /identity/v3/projects/d7fac6f013d94c59801313e73ae2f3c0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:34.109662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033d5d-3e8f-407d-8990-b996a3e2c618 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.109662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033d5d-3e8f-407d-8990-b996a3e2c618 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.109662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033d5d-3e8f-407d-8990-b996a3e2c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.128188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-34033d5d-3e8f-407d-8990-b996a3e2c618 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:34.129431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34033d5d-3e8f-407d-8990-b996a3e2c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:34.137771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1005/1984] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:34.150964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.151528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.151901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.171985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:34.173258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:34.183182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1006/1985] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:34.187165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.196535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d182ea8-df79-467c-b1e7-1f9f489c8577 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.201706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.202407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.202692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.203009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.203664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:47:34.204241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.213000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.218357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1007/1986] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:47:34.227470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.239287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e463-a642-4106-a329-50bcf74d8765 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.240964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:34.244738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.247336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.247336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.247336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.247336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.248571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.345053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1008/1987] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:34.346563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.349177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21cbf2237b4246dbb8b0ea02fdd1cbdb'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.361148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21cbf2237b4246dbb8b0ea02fdd1cbdb'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.368836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21cbf2237b4246dbb8b0ea02fdd1cbdb'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.397310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:34.422105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d21e5-8f9c-4be5-8f97-7816dc3c9c89 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.428135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.428937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.429312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.432003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.432003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.432826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.485890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21cbf2237b4246dbb8b0ea02fdd1cbdb'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.487093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fb6b8065-a2e1-4333-aba6-c8ce0575378c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21cbf2237b4246dbb8b0ea02fdd1cbdb'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:34.493862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 980/1988] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:33 2022] DELETE /identity/v3/users/21cbf2237b4246dbb8b0ea02fdd1cbdb => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:34.537204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.537491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.537771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.552238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1009/1989] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/users => generated 282 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:34.556473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.558763 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:34.561677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] MFA Rules not processed for user `60d10ce14ee3486f8e714dcc912d1527`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:34.562933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:34.578149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7b707-a881-4feb-8fbe-1c000ecf333d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.643478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 981/1990] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:34.650458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.651396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.651862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.652312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.653306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.655868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.677476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1010/1991] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:34.681536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.705533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f227-c313-41b4-9eec-7dddd98b7bd8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.709402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.710025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.710349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.710601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.711533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.714195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d3c7c-2089-4393-9141-6e1d95a54055 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.731657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 982/1992] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:47:34.731807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:34.742794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38e244-ac9b-47c2-a3fb-ada98c81cda6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:34.747656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:34.748302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.748578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.748851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] PATH_INFO: `/v3/projects/245fc4f5d7a3450c8b17b56c4f3e18ef/users/e7ff98718aab44fd8ee26190c86da7d3/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.766388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=245fc4f5d7a3450c8b17b56c4f3e18ef, user_id=e7ff98718aab44fd8ee26190c86da7d3, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:34.769782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:34.846173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1011/1993] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:34 2022] PUT /identity/v3/projects/245fc4f5d7a3450c8b17b56c4f3e18ef/users/e7ff98718aab44fd8ee26190c86da7d3/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:34.857908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:34.860459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:34.860459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:34.906232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] MFA Rules not processed for user `e7ff98718aab44fd8ee26190c86da7d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:34.910033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:35.041158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 983/1994] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:35.045647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.100127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.102114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:35.145172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:35.166923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c4d03-c89b-403d-b26f-11cb29bba74d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.187594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.188425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.188854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.191455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.191455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.193785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.215892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0519b19-ae39-4937-a568-9e452c953742 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.222953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.223770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.224113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.224445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] PATH_INFO: `/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.225794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6b32c2e0404fe7aeb7941e5f0abb7c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.228207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.319784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1012/1995] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/projects => generated 363 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:35.320788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.353449 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:35.378769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6b32c2e0404fe7aeb7941e5f0abb7c', 'request_id': 'req-9c77093e-3132-4cd5-a7e4-7a7683431e4b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:35.398613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80afe0-88c6-4f7a-8240-c9863beae4e5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.465972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.466265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6b32c2e0404fe7aeb7941e5f0abb7c', 'request_id': 'req-9c77093e-3132-4cd5-a7e4-7a7683431e4b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:35.466603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.466883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.467154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.467815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.469874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.476414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6b32c2e0404fe7aeb7941e5f0abb7c', 'request_id': 'req-9c77093e-3132-4cd5-a7e4-7a7683431e4b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:35.488202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] The token cache is being invalidate because project 4a6b32c2e0404fe7aeb7941e5f0abb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:35.488541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:35.489216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9c77093e-3132-4cd5-a7e4-7a7683431e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:35.490794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 984/1996] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:35 2022] DELETE /identity/v3/projects/4a6b32c2e0404fe7aeb7941e5f0abb7c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:35.495570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086e10-cda0-4553-9df7-cb71c845e7e9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.496521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086e10-cda0-4553-9df7-cb71c845e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.496521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086e10-cda0-4553-9df7-cb71c845e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.517196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ff086e10-cda0-4553-9df7-cb71c845e7e9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:35.521980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ff086e10-cda0-4553-9df7-cb71c845e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:35.527590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 985/1997] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:35.536318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.536600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.536869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.568266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:35.569481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:35.577797 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 986/1998] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:35.581887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.591980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4b2a5-d499-4474-ac83-e88c2f58d839 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.596820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.597425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.597741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.598027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.598663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:47:35.598967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.600556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.617777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 987/1999] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:47:35.622314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.624089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1013/2000] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:47:35.632998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.635671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:35.640086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f9760-354e-4cac-8787-d52ae9304e6a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.659524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fdd17-85e6-4680-9ae7-3e336da4b14b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.667843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.668534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.671174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.671453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.672205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.673810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.677399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.678047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.678877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.679851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.681412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.683896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.692356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1014/2001] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:47:35.694522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.706921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74c39d-010e-4738-953d-cf953ba68393 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:35.733382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:35.734740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:35.735032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:35.735291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] PATH_INFO: `/v3/projects/5858111a779f46f897ab1e1845f95ddd/users/46b2153cc2aa420aacf4b05b2aa3f3a4/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:35.768374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5858111a779f46f897ab1e1845f95ddd, user_id=46b2153cc2aa420aacf4b05b2aa3f3a4, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:35.780944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:35.921912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 988/2002] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/projects => generated 383 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:35.923350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.982189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1015/2003] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:35 2022] PUT /identity/v3/projects/5858111a779f46f897ab1e1845f95ddd/users/46b2153cc2aa420aacf4b05b2aa3f3a4/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:35.986219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:35.999684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66664f-0811-49da-ab08-c8fc8714e1a1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.016639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:36.074167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.075616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.075891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.076338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.076940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.079089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c48517-29bc-492b-8aa3-7d92f4577c1e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.079591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.085822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.086463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.086741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.087022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.087667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.089251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.131365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1016/2004] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:36.135720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.147962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32b845-8763-476f-861f-cfe17d585f3b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.156592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.157955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.158271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.158660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] PATH_INFO: `/v3/projects/5858111a779f46f897ab1e1845f95ddd/users/46b2153cc2aa420aacf4b05b2aa3f3a4/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.161564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5858111a779f46f897ab1e1845f95ddd, user_id=46b2153cc2aa420aacf4b05b2aa3f3a4, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.164627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3108bb-87fd-43fa-8e45-814834c63a53 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.172089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 989/2005] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:35 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:36.182098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.193385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1017/2006] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:36 2022] PUT /identity/v3/projects/5858111a779f46f897ab1e1845f95ddd/users/46b2153cc2aa420aacf4b05b2aa3f3a4/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:36.197234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec70759-0059-4e58-97c6-263956893a66 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.211636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.211919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.212196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.235475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:36.246259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] MFA Rules not processed for user `46b2153cc2aa420aacf4b05b2aa3f3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:36.247852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:36.301423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.302112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.302436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.303702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.304531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.309587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.332752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 990/2007] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:36.339376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.356793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878165-e9a4-4d21-937a-f23bc9fd626f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.366329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.368569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.369337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.369817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.372569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.375100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1018/2008] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:36.376727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.390322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.396335 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 991/2009] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:36.396458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:36.414732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.428208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b903-07b1-4252-9faa-47863a4d2b65 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.438172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.439023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.439905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.439905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] PATH_INFO: `/v3/projects/6dd61a88e563425fadd342700b64d1b6/users/325f361d93e7486e85520d2226811121/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.461956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd61a88e563425fadd342700b64d1b6, user_id=325f361d93e7486e85520d2226811121, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.465377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.488883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b149a-15c0-4d5f-9900-b5e3d425176b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.492732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.493839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.494347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.494774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.505271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.506795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ef054-bdc0-4567-baa5-741b6f373351 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.516319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1019/2010] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:47:36.600709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 992/2011] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:36 2022] PUT /identity/v3/projects/6dd61a88e563425fadd342700b64d1b6/users/325f361d93e7486e85520d2226811121/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:36.618396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.618685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.618966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.647427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] MFA Rules not processed for user `325f361d93e7486e85520d2226811121`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:36.648627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:36.730720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1020/2012] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:36.754028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.755873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:36.815239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0e47d-fac2-456f-b1f5-225a2621271b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.822278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.823000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.823498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.823813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.835623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.837264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.847990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:36.848746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 993/2013] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:36.859484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ed640-cf0b-4d54-94be-6f45ea974f94 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:36.867992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:36.868877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:36.869338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:36.869833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:36.872067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:36.927525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:36.964382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:36.965883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40cb8-b188-479a-a0c6-5d04cf84dc57 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:36.975631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1021/2014] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:47:41.024589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:41.038689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:41.094283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88289-5427-4c44-8e64-e2f7620b7dd3 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:41.099174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:41.099814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:41.100093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:41.100415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] PATH_INFO: `/v3/users/e7ff98718aab44fd8ee26190c86da7d3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:41.101760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ff98718aab44fd8ee26190c86da7d3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:41.103634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:41.329534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ff98718aab44fd8ee26190c86da7d3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:41.341647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ff98718aab44fd8ee26190c86da7d3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:41.349046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ff98718aab44fd8ee26190c86da7d3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:41.456470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ff98718aab44fd8ee26190c86da7d3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:41.457137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ff98718aab44fd8ee26190c86da7d3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:41.463459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 994/2015] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:41 2022] DELETE /identity/v3/users/e7ff98718aab44fd8ee26190c86da7d3 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:41.468134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:41.468783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:41.468783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:41.490001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:41.500057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:41.559686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1022/2016] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:41.569540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:41.572717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:41.645011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d94d-8b15-4698-baca-a2fe21610b66 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:41.649092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:41.649693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:41.649968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:41.650263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:41.659993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:41.661421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:41.671065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 995/2017] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:42.301265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:42.302743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:42.316951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648ad1-eeb0-4691-a4c5-7b4689a9c1f5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:42.326818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:42.326818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:42.326818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:42.326818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] PATH_INFO: `/v3/projects/245fc4f5d7a3450c8b17b56c4f3e18ef` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:42.327130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245fc4f5d7a3450c8b17b56c4f3e18ef)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:42.329430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:42.474643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245fc4f5d7a3450c8b17b56c4f3e18ef', 'request_id': 'req-3ae7201d-c835-4ff0-99a5-ce479f779583'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.481119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245fc4f5d7a3450c8b17b56c4f3e18ef', 'request_id': 'req-3ae7201d-c835-4ff0-99a5-ce479f779583'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.487472 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245fc4f5d7a3450c8b17b56c4f3e18ef', 'request_id': 'req-3ae7201d-c835-4ff0-99a5-ce479f779583'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.591766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:42.593219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:42.608502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366077b9-ce83-4b34-9ed5-fbd8396ee6bd service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:42.612490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] The token cache is being invalidate because project 245fc4f5d7a3450c8b17b56c4f3e18ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:42.612786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.613431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.614938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1023/2018] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:42 2022] DELETE /identity/v3/projects/245fc4f5d7a3450c8b17b56c4f3e18ef => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:42.622835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:42.624186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:42.639410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:42.666315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:42.667225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:42.667225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:42.667512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] PATH_INFO: `/v3/users/d614a216aa0c42a983af191117871a08` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:42.668639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d614a216aa0c42a983af191117871a08)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:42.670622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:42.677007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:42.694599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7201d-c835-4ff0-99a5-ce479f779583 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:42.697192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:42.697827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:42.698137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:42.698413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] PATH_INFO: `/v3/users/46b2153cc2aa420aacf4b05b2aa3f3a4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:42.699575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b2153cc2aa420aacf4b05b2aa3f3a4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:42.701549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:42.853693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd614a216aa0c42a983af191117871a08'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.869372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd614a216aa0c42a983af191117871a08'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.876547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd614a216aa0c42a983af191117871a08'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.884377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b2153cc2aa420aacf4b05b2aa3f3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.896404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b2153cc2aa420aacf4b05b2aa3f3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.903239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b2153cc2aa420aacf4b05b2aa3f3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.903928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b2153cc2aa420aacf4b05b2aa3f3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.958747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd614a216aa0c42a983af191117871a08'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.959371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd614a216aa0c42a983af191117871a08'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:42.965823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 996/2019] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:42 2022] DELETE /identity/v3/users/d614a216aa0c42a983af191117871a08 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:43.010252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b2153cc2aa420aacf4b05b2aa3f3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.016954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1024/2020] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:42 2022] DELETE /identity/v3/users/46b2153cc2aa420aacf4b05b2aa3f3a4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:43.214607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.216150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.297104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf271c3-33fb-494d-b6c3-cb1f81261df9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:43.358730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:43.359524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.359813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.360091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] PATH_INFO: `/v3/projects/be8191545f754abbbb1cf75b94c1961a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.361358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8191545f754abbbb1cf75b94c1961a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:43.363458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:43.365240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:43.524836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8191545f754abbbb1cf75b94c1961a', 'request_id': 'req-ef5a9b2a-0adc-431d-9e9e-5c105084f435'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.543620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.545862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.562460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0271007-879d-4608-ab22-16e9146c674c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:43.623889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:43.624516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.624794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.625067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] PATH_INFO: `/v3/projects/5858111a779f46f897ab1e1845f95ddd` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.626305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5858111a779f46f897ab1e1845f95ddd)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:43.628821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:43.633006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8191545f754abbbb1cf75b94c1961a', 'request_id': 'req-ef5a9b2a-0adc-431d-9e9e-5c105084f435'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.639025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8191545f754abbbb1cf75b94c1961a', 'request_id': 'req-ef5a9b2a-0adc-431d-9e9e-5c105084f435'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.651650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] The token cache is being invalidate because project be8191545f754abbbb1cf75b94c1961a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:43.651942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.652593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef5a9b2a-0adc-431d-9e9e-5c105084f435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.654121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 997/2021] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:43 2022] DELETE /identity/v3/projects/be8191545f754abbbb1cf75b94c1961a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:43.658868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31805-c9e0-4ade-94ba-f3940f82de92 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.659155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31805-c9e0-4ade-94ba-f3940f82de92 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.659433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31805-c9e0-4ade-94ba-f3940f82de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.679572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c3d31805-c9e0-4ade-94ba-f3940f82de92 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:43.680815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c3d31805-c9e0-4ade-94ba-f3940f82de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.688714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 998/2022] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:43.696543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.696867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.697140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.716712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:43.717889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.726791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 999/2023] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:43.730670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.739926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baa460-41f2-4574-813a-a105c26f6e89 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:43.748547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5858111a779f46f897ab1e1845f95ddd', 'request_id': 'req-f7739bed-efab-4102-966c-77cf5b2e15a2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.756091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5858111a779f46f897ab1e1845f95ddd', 'request_id': 'req-f7739bed-efab-4102-966c-77cf5b2e15a2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.762250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5858111a779f46f897ab1e1845f95ddd', 'request_id': 'req-f7739bed-efab-4102-966c-77cf5b2e15a2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.801520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:43.802400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.802681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.802954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.803598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:47:43.803906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:43.805575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:43.817657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1000/2024] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:47:43.823229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.833636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80e13-7071-43ae-9014-b4caa7ff86f4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:43.838562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:43.839261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.839551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.839826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.840526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:43.842477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:43.842724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:43.846516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] The token cache is being invalidate because project 5858111a779f46f897ab1e1845f95ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:43.846917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.847946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f7739bed-efab-4102-966c-77cf5b2e15a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:43.853649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1025/2025] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:43 2022] DELETE /identity/v3/projects/5858111a779f46f897ab1e1845f95ddd => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:43.856466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4109c-0b12-4a41-bdc9-efc275e26beb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.859574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4109c-0b12-4a41-bdc9-efc275e26beb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.859574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4109c-0b12-4a41-bdc9-efc275e26beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.882292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-26d4109c-0b12-4a41-bdc9-efc275e26beb None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:43.886005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-26d4109c-0b12-4a41-bdc9-efc275e26beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.902781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1026/2026] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:43.911615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:43.911903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:43.912189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:43.934197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:43.935303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:43.944406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1027/2027] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:43.948833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.956526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1001/2028] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:43.957853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:43.959294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431a950-5edb-48a7-a91a-ef5704897d86 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:43.959733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:44.015102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:44.019603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.020325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.020671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.021019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.021757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:47:44.022165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.023833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.032228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb03b-35aa-4f53-a16b-2f2453a39a29 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.037131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.037821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1028/2029] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:47:44.037936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.039924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.039924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.039924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.041550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.041980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.053690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b633862-66f3-4cdb-974e-3026b4f879eb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.059352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.059961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.060241 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.060517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.061194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.063148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.178801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1002/2030] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:43 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:44.179487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1029/2031] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:47:44.180260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.180398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.191092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63723c-fd25-4147-9017-af2618802f4d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.247061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faf755-d9bb-4d0d-8b75-4e96e91f4b31 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.253720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.253881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.253999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.254098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.254174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.254243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.254326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.254410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.254481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.254551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.255610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.256085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.265890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1003/2032] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:47:44.270190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.279841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5275f-7161-4d28-8bcd-4d8e6317eed7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.284437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.285013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.285301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.285568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.286398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.287958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.297677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1004/2033] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:44.302294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.311876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee47e-a26a-4295-a66d-3ad89cf702f0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.317064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.317677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.317960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.318268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] PATH_INFO: `/v3/projects/f02e440a52824687b246cb75b4f135cd/users/b193d0924fb448bfac2027034cb8dfe5/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.333224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02e440a52824687b246cb75b4f135cd, user_id=b193d0924fb448bfac2027034cb8dfe5, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.335886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452a87-4083-4575-84cd-17779e56996f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.353367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1030/2034] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:44.357393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.367940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca905a-4c0a-4e9a-ae2d-1209c8cb62f7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.372637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.373227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.373508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.373808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.374492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.376070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.386518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1031/2035] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:44.390492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.400064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ab778-b5f5-4fe8-aa88-c2c636bbc2bb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.404646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.405214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.405495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.406384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.406481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.408469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.419069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1032/2036] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:44.422876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.431676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1005/2037] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:44 2022] PUT /identity/v3/projects/f02e440a52824687b246cb75b4f135cd/users/b193d0924fb448bfac2027034cb8dfe5/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:44.433895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30791a-9a6d-48ec-aa4d-351e2f3715ec admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.436564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.436847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.437123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.454088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:44.455423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:44.456624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:44.495702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.497729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.498019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.498326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] PATH_INFO: `/v3/projects/64df40be0f274f9b805ce7f693150030/users/23ce2354a95148568473a5afbe6cdafc/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.514360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64df40be0f274f9b805ce7f693150030, user_id=23ce2354a95148568473a5afbe6cdafc, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.517052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e62476-819a-4b99-aa24-ad7f29d6019b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.522563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1006/2038] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:44.533712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.535919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:44.586598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:44.599295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e36a-2acb-4c83-b1f5-05dafd3512e2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.601146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1033/2039] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:44 2022] PUT /identity/v3/projects/64df40be0f274f9b805ce7f693150030/users/23ce2354a95148568473a5afbe6cdafc/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:47:44.611773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.612135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.612477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.644054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] MFA Rules not processed for user `23ce2354a95148568473a5afbe6cdafc`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:44.645375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:44.659638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.660256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.660534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.660805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.712146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1034/2040] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:44.716637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.722622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.724078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.727517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bda4a-77be-42f3-b973-921585cffa96 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.732189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.732778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.733059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.733333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.733755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1007/2041] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:47:44.733965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.736594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:44.863791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1035/2042] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:47:44.868233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:44.878273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161493-6cd5-44e7-8581-598019961706 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:44.956130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:44.956830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:44.957270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:44.957670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:44.958438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:44.960485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.050949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1008/2043] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:44 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:45.055045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:45.065422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571c979-1979-4fa0-8b8a-867672d4b5d6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:45.070100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:45.070694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.070970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.071239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.071887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:45.073463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.083983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1036/2044] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:47:45.099587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:45.112014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1ffdc-321c-4a3a-a38f-868623c7f88c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:45.117398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:45.118353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.118688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.119242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] PATH_INFO: `/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8/users/7d1b7aa66e3b426d9793e62f82c6c997/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.143152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c1780cf8be4a77ae10d96f59ba87c8, user_id=7d1b7aa66e3b426d9793e62f82c6c997, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:45.146895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.253824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1009/2045] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:45 2022] PUT /identity/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8/users/7d1b7aa66e3b426d9793e62f82c6c997/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:45.260390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:45.272696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956a95a-aa74-4ff1-838d-88dfdbeb1d1f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:45.335430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:45.336040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.336322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.336594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.337238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:45.338855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf48205-d9e4-4839-9892-9ddd243d7ff0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.344569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:45.351106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1037/2046] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:45.355092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:45.366515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ae4c1-2525-4b27-bf51-19602744934f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:45.374148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:45.374148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.374148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.374148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] PATH_INFO: `/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8/users/7d1b7aa66e3b426d9793e62f82c6c997/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.375722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c1780cf8be4a77ae10d96f59ba87c8, user_id=7d1b7aa66e3b426d9793e62f82c6c997, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:45.378822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.386746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:45.392577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1010/2047] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:45 2022] PUT /identity/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8/users/7d1b7aa66e3b426d9793e62f82c6c997/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:45.403154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f64fe-12f0-4c71-b14d-0830bbfc1f30 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.403439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f64fe-12f0-4c71-b14d-0830bbfc1f30 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.404327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f64fe-12f0-4c71-b14d-0830bbfc1f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.440332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b49f64fe-12f0-4c71-b14d-0830bbfc1f30 None None] MFA Rules not processed for user `7d1b7aa66e3b426d9793e62f82c6c997`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:45.441729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b49f64fe-12f0-4c71-b14d-0830bbfc1f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:45.510772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1038/2048] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:45.522110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:45.523864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:45.581736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3dd31-63ef-4f67-91bc-0282a1b7b1f4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:45.585671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:45.586313 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:45.586593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:45.586862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:45.652832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:45.654249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:45.663987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1011/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:51.079908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87fc61-a244-49bc-855e-178397e778a0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:51.080917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87fc61-a244-49bc-855e-178397e778a0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:51.081405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87fc61-a244-49bc-855e-178397e778a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:51.114234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1b87fc61-a244-49bc-855e-178397e778a0 None None] MFA Rules not processed for user `b193d0924fb448bfac2027034cb8dfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:51.115596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1b87fc61-a244-49bc-855e-178397e778a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:51.185069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1039/2050] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:51.199252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:51.206439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:51.290323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaacc2-f492-4892-90e7-1b710d5a8d93 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:51.296135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:51.296757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:51.297037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:51.297308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:51.306782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:51.308474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:51.324552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1012/2051] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:54.426297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:54.426720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:54.426887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:54.447960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] MFA Rules not processed for user `b193d0924fb448bfac2027034cb8dfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:54.449250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:54.458884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1040/2052] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:57.502789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:57.503921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:57.546929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:57.573534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b53a2-7b39-4749-a1e6-0c9ea2015e99 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:57.580466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:57.581106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:57.581387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:57.581661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] PATH_INFO: `/v3/users/325f361d93e7486e85520d2226811121` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:57.582840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] RBAC: Authorizing `identity:delete_user(user_id=325f361d93e7486e85520d2226811121)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:57.584880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:57.741938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:57.752183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2659f0b-770a-4272-8b17-be8b1fb63389 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:57.755734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:57.756327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:57.756604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:57.756873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:57.765895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:57.767380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671a78-ec37-4d89-bd36-ac4a1f0ff9ea service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:57.777056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1041/2053] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:47:57.801624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325f361d93e7486e85520d2226811121'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:57.818683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325f361d93e7486e85520d2226811121'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:57.833186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325f361d93e7486e85520d2226811121'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:57.926364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325f361d93e7486e85520d2226811121'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:57.927067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325f361d93e7486e85520d2226811121'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:57.944757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1013/2054] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:47:57 2022] DELETE /identity/v3/users/325f361d93e7486e85520d2226811121 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:57.945123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:57.945123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:57.945123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:57.963659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:57.964876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:58.019259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:58.030393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1042/2055] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:47:58.040969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:58.044220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:58.099658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaedf1-f6cd-48fc-a2a9-68c97096408d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:58.103229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:58.103824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:58.104102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:58.104371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:58.114392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:58.115833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156285d0-2127-4e72-bb0d-dd5c87a836fa service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:58.126039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1014/2056] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:47:58.656837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:58.661650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:58.674579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19cb1f-ccd9-46fd-8130-0c4a60507b69 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:58.679743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:58.680410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:58.680736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:58.680968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] PATH_INFO: `/v3/projects/6dd61a88e563425fadd342700b64d1b6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:58.682261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd61a88e563425fadd342700b64d1b6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:58.684183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:58.899795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd61a88e563425fadd342700b64d1b6', 'request_id': 'req-7c42d751-43eb-4642-bf83-304e7174d525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:58.906755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd61a88e563425fadd342700b64d1b6', 'request_id': 'req-7c42d751-43eb-4642-bf83-304e7174d525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:58.913618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd61a88e563425fadd342700b64d1b6', 'request_id': 'req-7c42d751-43eb-4642-bf83-304e7174d525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:59.047077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] The token cache is being invalidate because project 6dd61a88e563425fadd342700b64d1b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:47:59.047426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:59.048966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7c42d751-43eb-4642-bf83-304e7174d525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:47:59.053715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1043/2057] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:47:58 2022] DELETE /identity/v3/projects/6dd61a88e563425fadd342700b64d1b6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:59.062495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.062495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.062495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.097266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:59.098796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:59.182247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:47:59.198659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1015/2058] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:59.203969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.211602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.211602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.226407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:59.227629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:59.237569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1044/2059] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:47:59.241923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.252190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e76ff-1dde-497e-9cd2-759c3ecf123f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.256832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.257456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.257732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.258001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.258690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:47:59.258986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.260591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.272667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1016/2060] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:47:59.277098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.286798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df0714-a037-4a62-b854-8589c74bdacf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.291572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.292165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.292439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.292707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.293333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.295234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.436085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1045/2061] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:47:59 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:59.441145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.451363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75422a8-5d1c-4e01-92c7-c3f803b88dc8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.508784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.509549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.509839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.510150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.510799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.512951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.655979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1017/2062] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:47:59 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:47:59.657698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.668311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88478702-43bc-43f1-a7ad-cb35681fb551 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.673730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.674369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.674765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.675044 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.675696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.677337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.688192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1046/2063] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:47:59.691897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.702320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b3e4-332c-46eb-b3e5-9ad95a1d271a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.707159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.707756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.708058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.708389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.709042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.710665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd3d05-1822-4561-80ce-fa0344d3f1f4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.721605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1018/2064] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:47:59.725810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:47:59.736609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb660-b6e9-49c7-8475-da2b534a7944 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:47:59.741139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:47:59.741765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.742046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.742343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] PATH_INFO: `/v3/projects/81a2ef9a274a482eb2dcba0ef75c71bd/users/131229af575d4343b149cce473b4a5e6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.761517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a2ef9a274a482eb2dcba0ef75c71bd, user_id=131229af575d4343b149cce473b4a5e6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:47:59.763098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:47:59.878508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1047/2065] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:47:59 2022] PUT /identity/v3/projects/81a2ef9a274a482eb2dcba0ef75c71bd/users/131229af575d4343b149cce473b4a5e6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:47:59.889562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:47:59.889856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:47:59.890168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:47:59.918818 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] MFA Rules not processed for user `131229af575d4343b149cce473b4a5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:47:59.920040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:47:59.989577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1019/2066] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:47:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:00.010992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:00.013320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:00.072372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f296f99-5a8b-4597-a72b-d2cd77d42a45 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:00.075921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:00.082423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:00.082423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:00.082423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:00.086488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:00.087947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c56b-1526-4f15-bd79-c524c783266e service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:00.097400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1048/2067] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:00.123018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:00.194270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8c5f1-ee3c-4558-b6ec-bf744c687ff8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:00.201353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:00.202497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:00.202955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:00.204168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] PATH_INFO: `/v3/users/131229af575d4343b149cce473b4a5e6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:00.206047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=131229af575d4343b149cce473b4a5e6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:00.209638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:00.429047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131229af575d4343b149cce473b4a5e6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:00.441238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131229af575d4343b149cce473b4a5e6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:00.448321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131229af575d4343b149cce473b4a5e6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:00.504455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:00.526576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131229af575d4343b149cce473b4a5e6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:00.527168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131229af575d4343b149cce473b4a5e6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:00.534600 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1020/2068] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:00 2022] DELETE /identity/v3/users/131229af575d4343b149cce473b4a5e6 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:00.538914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:00.539271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:00.539610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:00.561586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:00.562836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:00.574860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:00.623457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1049/2069] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:48:00.633253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:00.634680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:00.691829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ceb81-33ef-4718-b0c5-393556d310aa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:00.695554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:00.696163 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:00.696440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:00.696729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:00.707840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:00.709353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196b7da-6196-4984-8b70-794a9e750aa4 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:00.720041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1021/2070] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:01.180286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.181944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:01.197064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04e72c-d3e5-4255-bf2e-b2689fd99821 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.202428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.203993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.204342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.204614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] PATH_INFO: `/v3/projects/81a2ef9a274a482eb2dcba0ef75c71bd` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.206282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a2ef9a274a482eb2dcba0ef75c71bd)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.208161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.318898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a2ef9a274a482eb2dcba0ef75c71bd', 'request_id': 'req-6dea46d9-4b23-496a-a200-67f16a6babef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:01.325616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a2ef9a274a482eb2dcba0ef75c71bd', 'request_id': 'req-6dea46d9-4b23-496a-a200-67f16a6babef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:01.331966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a2ef9a274a482eb2dcba0ef75c71bd', 'request_id': 'req-6dea46d9-4b23-496a-a200-67f16a6babef'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:01.422688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] The token cache is being invalidate because project 81a2ef9a274a482eb2dcba0ef75c71bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:01.423265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:01.424053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6dea46d9-4b23-496a-a200-67f16a6babef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:01.425586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1050/2071] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:01 2022] DELETE /identity/v3/projects/81a2ef9a274a482eb2dcba0ef75c71bd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:01.430514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.431032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.431307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.453936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:01.455304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:01.520491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1022/2072] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:01.528562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.528853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.529121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.550388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:01.551832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:01.560389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1051/2073] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:01.565059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.575655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea4bf9-8707-4c4a-acb8-43945e2ba417 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.580833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.581612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.581893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.582197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.582890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:48:01.583191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.584829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.598038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1023/2074] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:48:01.605524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.612661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a45f44-dba5-495f-8176-d4d90934326d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.617852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.618597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.619141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.619420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.620111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.622033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.726034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1052/2075] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:01 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:01.730696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.737664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:01.741723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e568412-b629-414a-8d55-626d244e6b35 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.800785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.801410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.801699 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.802025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.802708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.805115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.900949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1024/2076] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:01 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:01.905242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.915570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b62b9-a25d-4202-b6a1-0b68d6dd101f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.920109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.920710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.920989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.921263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.922011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.923950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.933801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1053/2077] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:01.938423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.948560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9b44d-8368-4475-b850-6b7389b74fbe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.953833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.954515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.954787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.955048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:01.955680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:01.957290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b412-ab14-48ff-9ae3-4b3a121a9122 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:01.967639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1025/2078] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:01.971748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:01.981695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817ffc-84ca-445c-b530-a24a8e85b1c2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:01.986831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:01.987474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:01.987744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:01.988012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] PATH_INFO: `/v3/projects/54b1322210eb4090abd9df79d186f56f/users/620abd54299148efac17cfa49826333a/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:02.002756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b1322210eb4090abd9df79d186f56f, user_id=620abd54299148efac17cfa49826333a, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:02.005390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:02.071155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1054/2079] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:01 2022] PUT /identity/v3/projects/54b1322210eb4090abd9df79d186f56f/users/620abd54299148efac17cfa49826333a/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:02.081438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:02.081714 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:02.082672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:02.109891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] MFA Rules not processed for user `620abd54299148efac17cfa49826333a`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:02.111119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:02.177075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1026/2080] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:02.187778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:02.189480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:02.231911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:02.246764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc1b51-3462-48a3-8b3f-bb8a65839680 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:02.250462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:02.251162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:02.251446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:02.251718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:02.259803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:02.261251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba169d-3848-436c-9d75-47b9bc95ffa7 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:02.270498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1055/2081] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:12.174254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:12.400090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=9 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:16.953258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:16.954823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:17.021991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dcd9a-6388-4c48-bfee-b39a7500bc1c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:17.026514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:17.028493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:17.028779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:17.029057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] PATH_INFO: `/v3/users/4ca49497e7cd43f48ff437eabdeea708` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:17.030328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca49497e7cd43f48ff437eabdeea708)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:17.032160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:17.231605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca49497e7cd43f48ff437eabdeea708'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:17.244820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca49497e7cd43f48ff437eabdeea708'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:17.252735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca49497e7cd43f48ff437eabdeea708'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:17.336762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca49497e7cd43f48ff437eabdeea708'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:17.337409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca49497e7cd43f48ff437eabdeea708'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:17.344574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1027/2082] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:16 2022] DELETE /identity/v3/users/4ca49497e7cd43f48ff437eabdeea708 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:48:17.349188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:17.349701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:17.349799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:17.381448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:17.383677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:17.452885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1056/2083] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:17.463142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:17.464570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:17.522884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541595-fd9e-47fd-8ebb-6b44dd91d83a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:17.526651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:17.527265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:17.527544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:17.527814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:17.538019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:17.539943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f907-e910-4644-b8f9-0f1a0da67eb2 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:17.550459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1028/2084] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:18.115876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:18.117563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:18.131521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068ba4-4ea1-439f-ac15-7a32ed618526 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:18.136308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:18.136937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.137217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.137489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] PATH_INFO: `/v3/projects/c27ebd8558f641a785c7da8d4751dfb8` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.139554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27ebd8558f641a785c7da8d4751dfb8)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:18.141555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:18.286858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27ebd8558f641a785c7da8d4751dfb8', 'request_id': 'req-73f7b875-abea-4e02-919b-555dab89fdfe'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:18.293375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27ebd8558f641a785c7da8d4751dfb8', 'request_id': 'req-73f7b875-abea-4e02-919b-555dab89fdfe'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:18.299848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27ebd8558f641a785c7da8d4751dfb8', 'request_id': 'req-73f7b875-abea-4e02-919b-555dab89fdfe'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:18.444388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] The token cache is being invalidate because project c27ebd8558f641a785c7da8d4751dfb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:18.445055 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:18.445291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73f7b875-abea-4e02-919b-555dab89fdfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:18.446838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1057/2085] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:18 2022] DELETE /identity/v3/projects/c27ebd8558f641a785c7da8d4751dfb8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:18.451708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a7919-7691-421a-bb12-955501538532 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.452160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a7919-7691-421a-bb12-955501538532 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.452254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a7919-7691-421a-bb12-955501538532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.505652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d65a7919-7691-421a-bb12-955501538532 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:18.506894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d65a7919-7691-421a-bb12-955501538532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:18.524025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:18.567263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1029/2086] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:18.574942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.575232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.575506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.596998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:18.598481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:18.607497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1058/2087] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:18.612451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a7919-7691-421a-bb12-955501538532 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:18.626332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a7919-7691-421a-bb12-955501538532 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:18.629009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:18.629618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.629900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.630207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.630949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:48:18.631257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:18.632995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:18.645936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1030/2088] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:48:18.649817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:18.661371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea30ec-a14a-4e96-b365-f8b1708b4439 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:18.666018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:18.666649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.666924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.667192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.667815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:18.669704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:18.820762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1059/2089] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:18 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:18.825131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:18.836195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b95c0-62e7-4882-ae81-5506cf9774f5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:18.899148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:18.899765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:18.900053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:18.900327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:18.900962 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:18.902919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:19.002737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1031/2090] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:18 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:19.006796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:19.018683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5889b3-7dc5-4205-b59c-6df37d7cacdb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:19.023842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:19.025226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:19.025568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:19.025959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:19.026645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:19.028258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:19.038889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1060/2091] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:19.042918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:19.054556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103e03-54b5-48cc-ac8a-949df61e2085 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:19.059146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:19.059761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:19.060037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:19.060305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:19.061608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:19.063256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e1635-d8b6-47c4-b7c0-fd76959fc1ef admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:19.073729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1032/2092] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:19.077828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:19.087825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a8581-300e-43a8-8997-59c63c059aec admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:19.092535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:19.093172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:19.093452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:19.093729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] PATH_INFO: `/v3/projects/aba6e95f376843d29b7fee2ef2b1b869/users/1c6f49dbafbc41fc989d4d12acc0a28c/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:19.108467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba6e95f376843d29b7fee2ef2b1b869, user_id=1c6f49dbafbc41fc989d4d12acc0a28c, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:19.111309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:19.198996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1061/2093] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:19 2022] PUT /identity/v3/projects/aba6e95f376843d29b7fee2ef2b1b869/users/1c6f49dbafbc41fc989d4d12acc0a28c/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:19.209970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b57-155c-484e-befc-c2d811b8471a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:19.210285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b57-155c-484e-befc-c2d811b8471a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:19.210562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f2b57-155c-484e-befc-c2d811b8471a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:19.238337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8c3f2b57-155c-484e-befc-c2d811b8471a None None] MFA Rules not processed for user `1c6f49dbafbc41fc989d4d12acc0a28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:19.239558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8c3f2b57-155c-484e-befc-c2d811b8471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:19.303097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1033/2094] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:19.313917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:19.315640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:19.337652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:19.374319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34994d24-501a-4f65-a9fb-d99fcaa305e9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:19.378828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:19.379523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:19.379861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:19.380160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:19.388983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:19.390497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:19.399414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1062/2095] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:23.206345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837511e-f8c2-4797-90e6-d45940435c6e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:23.206860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837511e-f8c2-4797-90e6-d45940435c6e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:23.206860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837511e-f8c2-4797-90e6-d45940435c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:23.238014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6837511e-f8c2-4797-90e6-d45940435c6e None None] MFA Rules not processed for user `1c6f49dbafbc41fc989d4d12acc0a28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:23.242466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6837511e-f8c2-4797-90e6-d45940435c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:23.250875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1034/2096] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:29.433120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=17 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:32.723707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:32.725109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:32.799953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754a17-e3d8-4f3a-92a6-897a78e60d3e service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:32.804737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:32.805434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:32.805713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:32.805984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] PATH_INFO: `/v3/users/620abd54299148efac17cfa49826333a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:32.807140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=620abd54299148efac17cfa49826333a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:32.809031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:33.047978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620abd54299148efac17cfa49826333a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:33.066894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620abd54299148efac17cfa49826333a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:33.073855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620abd54299148efac17cfa49826333a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:33.074636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620abd54299148efac17cfa49826333a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:33.175580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620abd54299148efac17cfa49826333a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:33.184505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1063/2097] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:32 2022] DELETE /identity/v3/users/620abd54299148efac17cfa49826333a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:33.196326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:33.197518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:33.198655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:33.222988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:33.223201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:33.228481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:33.280799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1035/2098] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:33.290672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:33.292520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:33.357910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156949d-7f8c-4c7f-bafd-13e6b2a61d6d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:33.357910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:33.357910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:33.357910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:33.358605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:33.367184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:33.368626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb60fb-9314-4d0c-a887-803cdd8cf0d6 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:33.382192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1064/2099] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:34.039285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:34.039285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:34.043741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd874f-e884-43ba-821e-107a947a3789 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:34.057759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:34.059179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.060121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.061374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] PATH_INFO: `/v3/projects/54b1322210eb4090abd9df79d186f56f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.063511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b1322210eb4090abd9df79d186f56f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:34.066749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:34.250783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b1322210eb4090abd9df79d186f56f', 'request_id': 'req-78285e37-5034-422d-8b19-b19ebca15076'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:34.392892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b1322210eb4090abd9df79d186f56f', 'request_id': 'req-78285e37-5034-422d-8b19-b19ebca15076'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:34.400514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b1322210eb4090abd9df79d186f56f', 'request_id': 'req-78285e37-5034-422d-8b19-b19ebca15076'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:34.412479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] The token cache is being invalidate because project 54b1322210eb4090abd9df79d186f56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:34.412790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:34.413477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-78285e37-5034-422d-8b19-b19ebca15076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:34.415037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1036/2100] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:34 2022] DELETE /identity/v3/projects/54b1322210eb4090abd9df79d186f56f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:34.420789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.421534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.421656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.456548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:34.458496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:34.480087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:34.532087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1065/2101] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:34.540038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.540326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.545113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.601142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:34.602398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:34.611698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1037/2102] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:34.615744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:34.626541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3a97-24a2-4dfb-bd64-b0ed1c014533 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:34.631448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:34.632070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.632350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.632623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.633281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:48:34.633593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:34.635216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:34.648183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1066/2103] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:48:34.652619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:34.671010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed923d11-8a4b-4a86-80e1-f4c5922fcb17 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:34.675794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:34.676713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.676994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.677259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.677901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:34.679829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:34.805694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1038/2104] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:34 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:34.810288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:34.828833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701c36e-c202-41a2-bddf-24e4d42d07e0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:34.895987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:34.896604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:34.896880 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:34.897148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:34.897894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:34.900655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:34.988249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1067/2105] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:34 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:34.992262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:35.020288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dee54-dd90-48f8-8d21-7d2a223285ba admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:35.028526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:35.029786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.030262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.030620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.031369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:35.033628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:35.050403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1039/2106] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:35.053811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:35.076678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d023c-f0bd-47e3-b775-7c5a98cd574f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:35.098813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:35.099723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.100184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.100667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.101573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:35.103417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0b536-44ff-4dca-91a3-f31ccc492d7b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:35.117697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1068/2107] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:35.122696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:35.139488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923294b-073e-4606-baf9-b6cbb82dcecb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:35.145174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:35.145836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.146157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.146441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] PATH_INFO: `/v3/projects/4395b654958545cc8913f4d7786c736a/users/b9a4f3995e1242de9d4bc87ce0608265/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.165337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4395b654958545cc8913f4d7786c736a, user_id=b9a4f3995e1242de9d4bc87ce0608265, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:35.169242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed101d0-8ccf-41f0-ac9b-586d7c7a592d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:35.223300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.223591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.223870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.243745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] MFA Rules not processed for user `41c9e5d537ab4a4a8931311d2dff3f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:35.244222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:35.269386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1040/2108] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:35 2022] PUT /identity/v3/projects/4395b654958545cc8913f4d7786c736a/users/b9a4f3995e1242de9d4bc87ce0608265/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:35.277168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.277525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.277907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.300288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:35.301847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:35.303380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1069/2109] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:48:35.334726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:35.375664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1041/2110] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:35.385767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:35.387217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:35.448166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144582c-4ed3-4d04-b54c-74dd085c077c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:35.503293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:35.503929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.504267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.504604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.514264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:35.515732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76d95e-0393-488f-a186-c36d5b24e5b8 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:35.527962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1070/2111] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:35.669829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:35.671798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:35.687248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbff24b-88b3-4ca5-a1e9-6d5c1049ef41 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:35.692824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:35.694156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:35.712534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:35.712534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] PATH_INFO: `/v3/users/23ce2354a95148568473a5afbe6cdafc` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:35.712534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ce2354a95148568473a5afbe6cdafc)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:35.712534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:36.068304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ce2354a95148568473a5afbe6cdafc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:36.095249 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ce2354a95148568473a5afbe6cdafc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:36.105012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ce2354a95148568473a5afbe6cdafc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:36.233643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ce2354a95148568473a5afbe6cdafc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:36.234377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ce2354a95148568473a5afbe6cdafc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:36.243425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1042/2112] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:35 2022] DELETE /identity/v3/users/23ce2354a95148568473a5afbe6cdafc => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:36.248578 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:36.249056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:36.249341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:36.319360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:36.330517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:36.473181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1071/2113] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:48:36.510648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:36.514046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:36.597765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545a12c-f11b-4454-b1f5-e5824bd21ea0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:36.601982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:36.602876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:36.603446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:36.603724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:36.631264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:36.633029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:36.662640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1043/2114] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:37.390039 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:37.391955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:37.405401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aa637-6ecd-492c-9078-75c88a77a52e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:37.411667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:37.412207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:37.412489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:37.412759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] PATH_INFO: `/v3/projects/64df40be0f274f9b805ce7f693150030` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:37.413913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64df40be0f274f9b805ce7f693150030)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:37.416550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:37.600385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64df40be0f274f9b805ce7f693150030', 'request_id': 'req-bb57f450-4dd5-42fd-803e-e76d818b1f5b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:37.608815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64df40be0f274f9b805ce7f693150030', 'request_id': 'req-bb57f450-4dd5-42fd-803e-e76d818b1f5b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:37.615079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64df40be0f274f9b805ce7f693150030', 'request_id': 'req-bb57f450-4dd5-42fd-803e-e76d818b1f5b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:37.698912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:37.777752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] The token cache is being invalidate because project 64df40be0f274f9b805ce7f693150030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:37.781478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:37.781478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:37.782378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1072/2115] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:37 2022] DELETE /identity/v3/projects/64df40be0f274f9b805ce7f693150030 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:37.784463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:37.800323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1235-28b2-44ff-9013-e99db8bebbc0 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:37.881830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:37.883067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:37.883534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:37.884195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] PATH_INFO: `/v3/users/7d1b7aa66e3b426d9793e62f82c6c997` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:37.886259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1b7aa66e3b426d9793e62f82c6c997)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:37.888779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:38.155008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1b7aa66e3b426d9793e62f82c6c997'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:38.166991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1b7aa66e3b426d9793e62f82c6c997'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:38.174694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1b7aa66e3b426d9793e62f82c6c997'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:38.247814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1b7aa66e3b426d9793e62f82c6c997'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:38.249372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d0d0ceae-a796-4903-a5b9-836c34b37dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1b7aa66e3b426d9793e62f82c6c997'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:38.257530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1044/2116] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:37 2022] DELETE /identity/v3/users/7d1b7aa66e3b426d9793e62f82c6c997 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:39.051989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:39.053838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:39.068310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f450-4dd5-42fd-803e-e76d818b1f5b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:39.133035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:39.133782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:39.134323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:39.136099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] PATH_INFO: `/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:39.138448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c1780cf8be4a77ae10d96f59ba87c8)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:39.145937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:39.267850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:39.327804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c1780cf8be4a77ae10d96f59ba87c8', 'request_id': 'req-5806c56b-9487-47c5-97b8-e234ebcd9045'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:39.338178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c1780cf8be4a77ae10d96f59ba87c8', 'request_id': 'req-5806c56b-9487-47c5-97b8-e234ebcd9045'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:39.352489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c1780cf8be4a77ae10d96f59ba87c8', 'request_id': 'req-5806c56b-9487-47c5-97b8-e234ebcd9045'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:39.494497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] The token cache is being invalidate because project 62c1780cf8be4a77ae10d96f59ba87c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:39.494984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:39.500027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5806c56b-9487-47c5-97b8-e234ebcd9045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:39.503046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1073/2117] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:39 2022] DELETE /identity/v3/projects/62c1780cf8be4a77ae10d96f59ba87c8 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:39.508015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:39.508312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:39.508593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:39.535875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:39.537296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:39.643736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1045/2118] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:39.651219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:39.651741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:39.652200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:39.694544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:39.703271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:39.704508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:39.723232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1074/2119] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:39.723416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:39.738531 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4e61b-8d3d-4911-b6a7-867fb12c8244 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:39.745516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:39.747020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:39.747484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:39.761427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:39.761427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:48:39.761427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:39.761427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:39.784474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1046/2120] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:48:39.789567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:39.814642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4261b-0c99-46a8-b3d9-a858a5f40c53 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:39.828511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:39.829372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:39.829860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:39.830356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:39.831208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:39.844590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:39.994138 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1075/2121] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:39 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:48:39.998663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.017807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9734f-0b85-4c01-8606-8c0322fc96a0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.110627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.112095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.112646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.112976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.113871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.120422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:40.253960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1047/2122] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:39 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:40.254260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.266848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71954a20-2b92-4c9d-9ac0-a8e583255031 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.282129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.287458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.287458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.287458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.287458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.287458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:40.297800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1076/2123] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:40.302014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.315540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2283-a824-4880-a5b3-89c449762a48 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.323603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.324530 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.324997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.325451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.326351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.328605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89495d5-71f4-4658-b406-4d566d690aa2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:40.342347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1048/2124] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:40.347264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.360157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aff3c-467a-403d-8b2d-c7da716ffc3f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.370263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.373437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.374575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.375093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] PATH_INFO: `/v3/projects/f986de2028d949e0841097b1c1238363/users/518f52b5d7ca44768b8cf1500852de5d/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.417146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f986de2028d949e0841097b1c1238363, user_id=518f52b5d7ca44768b8cf1500852de5d, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.421835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:40.546118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1077/2125] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:40 2022] PUT /identity/v3/projects/f986de2028d949e0841097b1c1238363/users/518f52b5d7ca44768b8cf1500852de5d/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:40.556555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.557026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.557465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.598751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] MFA Rules not processed for user `518f52b5d7ca44768b8cf1500852de5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:40.599989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:40.684778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1049/2126] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:40.685996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.755800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c1e21-b0b6-4b68-9a18-126982886d05 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.760491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.761101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.761383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.761659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.762872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.767366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:40.891573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1078/2127] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:40 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:40.895813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:40.908880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7dc1f-a82c-4b14-9ce8-257a9b83404c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:40.959812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:40.978740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:40.979385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:40.979668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:40.979939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:40.980641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:40.982613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.135394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1050/2128] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:40 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:48:41.136688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.147854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc71d1-f598-4625-a9fc-f97b4b7e48e1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.153397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.154002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.154320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.154597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.155276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:41.156875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.173043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1079/2129] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:41.174345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.185542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4de82a-1b93-4570-9be2-7005ed297b1a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.191297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.192412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.192757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.193101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] PATH_INFO: `/v3/projects/e6d928b940954c39bc207648a611376c/users/c3d326b9059048a29a2dbdc87262de8c/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.209941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d928b940954c39bc207648a611376c, user_id=c3d326b9059048a29a2dbdc87262de8c, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:41.212906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.326944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1051/2130] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:41 2022] PUT /identity/v3/projects/e6d928b940954c39bc207648a611376c/users/c3d326b9059048a29a2dbdc87262de8c/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:41.331015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.333312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:41.342897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df9ec4-5648-4b9d-80ca-eb772e8adafd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.401507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.402148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.402948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.403225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.403877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:41.405462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a199698-fbac-4d85-bf84-8a22efc17a13 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.418862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1080/2131] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:41.420213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.431066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4441c-fd8c-43f4-9aab-9593505dcc62 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.435887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.436627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.436968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.437430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] PATH_INFO: `/v3/projects/e6d928b940954c39bc207648a611376c/users/c3d326b9059048a29a2dbdc87262de8c/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.439916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d928b940954c39bc207648a611376c, user_id=c3d326b9059048a29a2dbdc87262de8c, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:41.442971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.455363 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1052/2132] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:41 2022] PUT /identity/v3/projects/e6d928b940954c39bc207648a611376c/users/c3d326b9059048a29a2dbdc87262de8c/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:41.468652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.470900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.470900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.514045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] MFA Rules not processed for user `c3d326b9059048a29a2dbdc87262de8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:41.515848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:41.596323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1081/2133] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:41.608574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.610646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:41.706406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1c08-17f8-45b9-bf3f-2d4e6dfdd06f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.710553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.711329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.711617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.711890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:41.725684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:41.728849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46765d77-5445-4b1c-bbaf-c4aed8e5e66b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:41.740881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1053/2134] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:41.883975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:41.904457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845e7dd-66f6-4baf-8ac2-9a9da1cdb303 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:41.918607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:41.920445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:41.920721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:41.922392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:42.022042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:42.023507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:42.033745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1082/2135] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:42.766715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:42.963828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:42.964122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:42.964401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:42.972358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:43.027072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] MFA Rules not processed for user `b9a4f3995e1242de9d4bc87ce0608265`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:43.029067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:43.150123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1054/2136] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:43.155801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.157425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:43.254581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c45f-a501-42f6-a1cb-7011ba1d26ae service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.260421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.262369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.262369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.263080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.263795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.265767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.430106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1083/2137] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:43 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:43.432819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.443547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042ad2-3282-4c34-b752-8ec3909a3cb6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.506511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.507315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.507595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.507871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.508508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.510489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.609598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1055/2138] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:43 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:43.613640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.624617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d5e8a-d72b-444a-a6cc-b6869f6c249f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.629201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.629796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.630108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.630387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.631033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.632626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.643624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1084/2139] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:43.647925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.658022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a41f80-faee-431d-a965-800a2c7e8e40 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.662679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.663326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.663605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.663878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] PATH_INFO: `/v3/projects/763412ca2259491ebfac445f6369c3f4/users/e4acdc49a7e0499db338bb50eec8d7ae/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.679318 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=763412ca2259491ebfac445f6369c3f4, user_id=e4acdc49a7e0499db338bb50eec8d7ae, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.682061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.757539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1056/2140] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:43 2022] PUT /identity/v3/projects/763412ca2259491ebfac445f6369c3f4/users/e4acdc49a7e0499db338bb50eec8d7ae/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:43.761509 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.772274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2e53-6691-4a81-8938-8c3d805c145b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.828658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.829265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.829572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.829847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.830902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.832490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34567d-0363-4922-9aaa-43ddd83e5323 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.842344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1085/2141] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:43.846687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:43.861081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9298-f622-4960-8877-481aa430b56b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:43.868644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:43.869303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:43.869593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:43.869914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] PATH_INFO: `/v3/projects/763412ca2259491ebfac445f6369c3f4/users/e4acdc49a7e0499db338bb50eec8d7ae/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:43.872339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763412ca2259491ebfac445f6369c3f4, user_id=e4acdc49a7e0499db338bb50eec8d7ae, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:43.875030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:43.881238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:43.892922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1057/2142] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:43 2022] PUT /identity/v3/projects/763412ca2259491ebfac445f6369c3f4/users/e4acdc49a7e0499db338bb50eec8d7ae/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:44.148523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb44049-d067-48ab-a948-893580ef378d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:44.149057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb44049-d067-48ab-a948-893580ef378d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:44.149474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb44049-d067-48ab-a948-893580ef378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:44.216307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-efb44049-d067-48ab-a948-893580ef378d None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:44.225797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-efb44049-d067-48ab-a948-893580ef378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:44.299890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:44.321271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1086/2143] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:48:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:44.380422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:44.384274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:44.499521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434562c9-a65a-49a9-94ee-43cd68be7fa4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:44.528106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:44.528736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:44.529013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:44.529281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:44.555117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:44.557131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba67aa9-eb13-4421-a5b8-13ff725d93db service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:44.598239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1058/2144] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:48:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:48:46.707291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb44049-d067-48ab-a948-893580ef378d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:46.896168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb44049-d067-48ab-a948-893580ef378d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:46.910632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:46.911963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:46.917118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:46.918272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] PATH_INFO: `/v3/users/518f52b5d7ca44768b8cf1500852de5d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:46.919948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518f52b5d7ca44768b8cf1500852de5d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:46.922934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:47.237493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518f52b5d7ca44768b8cf1500852de5d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:47.284514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518f52b5d7ca44768b8cf1500852de5d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:47.303872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518f52b5d7ca44768b8cf1500852de5d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:47.305901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518f52b5d7ca44768b8cf1500852de5d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:47.424586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518f52b5d7ca44768b8cf1500852de5d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:47.448431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1087/2145] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:46 2022] DELETE /identity/v3/users/518f52b5d7ca44768b8cf1500852de5d => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:47.449333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:47.449831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:47.449942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:47.494758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:47.497857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:47.659401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1059/2146] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:47.674133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:47.675977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:47.843548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00a9a-730b-4967-a397-a7a163800208 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:47.849465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:47.850648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:47.850934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:47.851207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:47.876354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:47.876354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:47.900192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1088/2147] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:48.818919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:48.821327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:48.882677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff737a-ca69-4a67-813e-f212b97265a2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:48.898959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:48.899860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:48.900302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:48.900739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] PATH_INFO: `/v3/projects/f986de2028d949e0841097b1c1238363` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:48.906716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f986de2028d949e0841097b1c1238363)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:48.909333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:49.118114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f986de2028d949e0841097b1c1238363', 'request_id': 'req-2f5976fe-700a-4f89-976a-850342b000b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.197366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f986de2028d949e0841097b1c1238363', 'request_id': 'req-2f5976fe-700a-4f89-976a-850342b000b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.212759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f986de2028d949e0841097b1c1238363', 'request_id': 'req-2f5976fe-700a-4f89-976a-850342b000b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.228974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] The token cache is being invalidate because project f986de2028d949e0841097b1c1238363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:49.230203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.230951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.234415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1060/2148] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:48 2022] DELETE /identity/v3/projects/f986de2028d949e0841097b1c1238363 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:49.240383 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:49.257479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953057f-7ef5-4fcb-a36e-339773a9208e service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:49.291871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:49.358170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:49.358915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:49.359263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:49.359611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] PATH_INFO: `/v3/users/c3d326b9059048a29a2dbdc87262de8c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:49.363380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d326b9059048a29a2dbdc87262de8c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:49.366485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:49.597860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d326b9059048a29a2dbdc87262de8c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.649161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d326b9059048a29a2dbdc87262de8c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.672011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d326b9059048a29a2dbdc87262de8c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.673602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d326b9059048a29a2dbdc87262de8c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.745490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-63f5e74a-f550-45c8-8e77-a1b271722bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d326b9059048a29a2dbdc87262de8c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:49.775398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1089/2149] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:48:49 2022] DELETE /identity/v3/users/c3d326b9059048a29a2dbdc87262de8c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:50.462458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:50.464682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:50.489660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5976fe-700a-4f89-976a-850342b000b5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:50.506769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:50.564309 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:50.565488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:50.565488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:50.565841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] PATH_INFO: `/v3/projects/e6d928b940954c39bc207648a611376c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:50.567241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d928b940954c39bc207648a611376c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:50.569219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:50.783811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d928b940954c39bc207648a611376c', 'request_id': 'req-a115189e-e376-40c6-9714-804a890ef7d4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:50.883210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d928b940954c39bc207648a611376c', 'request_id': 'req-a115189e-e376-40c6-9714-804a890ef7d4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:50.890767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d928b940954c39bc207648a611376c', 'request_id': 'req-a115189e-e376-40c6-9714-804a890ef7d4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:50.914239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] The token cache is being invalidate because project e6d928b940954c39bc207648a611376c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:48:50.914513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:50.915193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a115189e-e376-40c6-9714-804a890ef7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:48:50.925135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1061/2150] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:48:50 2022] DELETE /identity/v3/projects/e6d928b940954c39bc207648a611376c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:50.934580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:50.934580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:50.934580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:50.967336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:50.971052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:51.110094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1090/2151] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:51.116378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.116895 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.117355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.159291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:51.160542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:51.177655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1062/2152] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:51.184926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.195323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf72bd-8a10-4238-80f5-a86bb547e364 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.203378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.204010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.204341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.204619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.205280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:48:51.205584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.207368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:51.221702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1091/2153] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:48:51.225891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.235794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81221ae3-0885-4387-a151-41792b76ccb0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.243638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.244867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.245888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.246261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.246979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.249533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:51.360943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1063/2154] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:51 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:51.364723 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.378837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9711-c74d-4bae-b7f8-6854fb1399c6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.457863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:51.566151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.566941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.567299 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.567657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.568431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.574552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:51.727659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1092/2155] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:51 2022] POST /identity/v3/users => generated 284 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:48:51.728977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.744536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89348-2e74-4ee5-8c9c-f14c66cb7393 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.758889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.759729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.760129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.761572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.762675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.770156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:51.795152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1064/2156] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:51.796604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.807807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e50e5b-677a-40d6-ae8c-be893de964df admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.827730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.828594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.828948 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.829306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.830047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.831950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07e52-5050-4a87-80b1-8299d678ec73 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:51.857386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1093/2157] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:48:51.858640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:51.873658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b657a-8f82-4462-8dfe-e94779f6a5df admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:51.880521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:51.881465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:51.881812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:51.883717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] PATH_INFO: `/v3/projects/2ff97454a7e84c9d816254b1538233dd/users/9153c67fa2b94a8eb1f050cc0c64e967/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:51.917401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff97454a7e84c9d816254b1538233dd, user_id=9153c67fa2b94a8eb1f050cc0c64e967, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:51.925311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:52.015295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1065/2158] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:51 2022] PUT /identity/v3/projects/2ff97454a7e84c9d816254b1538233dd/users/9153c67fa2b94a8eb1f050cc0c64e967/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:48:52.029555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:52.030132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:52.030132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:52.101881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] MFA Rules not processed for user `9153c67fa2b94a8eb1f050cc0c64e967`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:52.103423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:52.307279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1094/2159] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:52.315905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:52.321524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:52.326141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:52.463254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:52.544171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f1f56-eae9-402f-879f-58e8f9503406 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:52.565695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:52.566367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:52.566650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:52.571716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:52.572469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:52.583559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:52.603812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ffbf-8f2c-4975-829e-5a152945c3fa None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:52.621481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:52.622190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:52.622471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:52.622920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:52.625211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:52.645866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:52.647542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:52.663805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1095/2160] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:52.722633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1066/2161] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:48:52 2022] POST /identity/v3/projects => generated 365 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:48:52.726468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:52.738683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8253fa-a4b0-4fd3-b5be-2ec854545fe9 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:52.754601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:52.830564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:52.831345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:52.831866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:52.832295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:52.833411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:52.835525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:52.976747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1096/2162] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:48:52 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:48:52.980743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:52.992234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439bebb-e1a0-43b6-8b64-51b3b46c705a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:53.000941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:53.005303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:53.006702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:53.007490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:53.008914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:53.017217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:53.037960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:53.039111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1067/2163] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:53.060322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313909-12b4-4d5b-936a-a3771b70e87e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:53.068739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:53.069520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:53.069941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:53.070261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] PATH_INFO: `/v3/projects/1e78d847d86e4dbebd7371658b6a099f/users/c794dc7aabad4868bc59650afafa977e/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:53.106196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e78d847d86e4dbebd7371658b6a099f, user_id=c794dc7aabad4868bc59650afafa977e, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:53.109000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:53.219790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1097/2164] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:53 2022] PUT /identity/v3/projects/1e78d847d86e4dbebd7371658b6a099f/users/c794dc7aabad4868bc59650afafa977e/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:48:53.223583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:53.238849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde9e29-6adf-4b7f-b6a4-373e886a0e3a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:53.373509 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:53.384950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:53.385378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:53.387081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:53.389707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:53.391581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cccec5-6dc3-4dd3-88dc-16be649d9075 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:53.436940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:53.437210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1068/2165] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:48:53.455978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476dc79-66dd-4513-9070-3d519a2add67 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:53.478674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:53.480142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:53.480539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:53.480907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] PATH_INFO: `/v3/projects/1e78d847d86e4dbebd7371658b6a099f/users/c794dc7aabad4868bc59650afafa977e/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:53.484832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e78d847d86e4dbebd7371658b6a099f, user_id=c794dc7aabad4868bc59650afafa977e, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:53.489678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:53.513877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1098/2166] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:48:53 2022] PUT /identity/v3/projects/1e78d847d86e4dbebd7371658b6a099f/users/c794dc7aabad4868bc59650afafa977e/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:48:53.524122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:53.524414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:53.524873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:53.615317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] MFA Rules not processed for user `c794dc7aabad4868bc59650afafa977e`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:53.620429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:53.845070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1069/2167] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:48:53.864888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:53.871623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:54.077356 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a39e-7930-4761-b90c-ecb41e06293b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:54.095601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:54.096624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:54.097085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:54.097550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:54.127398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:54.131964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29ad2-815e-4689-96d9-0f0efe29c52a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:54.183237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1099/2168] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:54.671539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:54.703945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb43b4-1617-4fe4-ad07-9e0adea6ec74 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:54.736661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:54.737705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:54.738404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:54.738902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:54.803477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:54.922499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:54.923980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:54.944428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1070/2169] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:57.282916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80609-e1eb-473a-940c-6c7d4368f945 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:57.283423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80609-e1eb-473a-940c-6c7d4368f945 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:57.283550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80609-e1eb-473a-940c-6c7d4368f945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:57.343632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-43d80609-e1eb-473a-940c-6c7d4368f945 None None] MFA Rules not processed for user `e4acdc49a7e0499db338bb50eec8d7ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:57.345400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-43d80609-e1eb-473a-940c-6c7d4368f945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:57.504311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:57.519732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1100/2170] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:48:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:57.537466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:57.539155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:57.667839 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1170b-6cc5-40fc-b1ab-ed796df738f0 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:57.679704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:57.680635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:57.681137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:57.681504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:57.684455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:57.761151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:57.762649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:57.773482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1071/2171] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:48:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:48:58.156878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:58.157169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:58.157447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:58.179245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:48:58.180533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:58.245365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1101/2172] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:48:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:48:58.262780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:48:58.264763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:48:58.348018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04dab1-413d-4901-8d70-c05bc3aa73b7 service glance] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:48:58.379993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:48:58.380623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:48:58.380899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:48:58.381166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:48:58.395892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:48:58.397319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:48:58.399893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8862993-e298-4d17-8663-28090252e6d2 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:48:58.409725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1072/2173] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:48:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:05.297935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:05.433167 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44fae9-9e37-45e6-a1fa-d7a88e60f772 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:05.446787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:05.447680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:05.448149 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:05.448645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] PATH_INFO: `/v3/users/9153c67fa2b94a8eb1f050cc0c64e967` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:05.450139 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9153c67fa2b94a8eb1f050cc0c64e967)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:05.452295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:05.765702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9153c67fa2b94a8eb1f050cc0c64e967'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:05.832803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9153c67fa2b94a8eb1f050cc0c64e967'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:05.860936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9153c67fa2b94a8eb1f050cc0c64e967'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:05.862583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9153c67fa2b94a8eb1f050cc0c64e967'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:05.958931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9153c67fa2b94a8eb1f050cc0c64e967'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:05.972429 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1102/2174] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:05 2022] DELETE /identity/v3/users/9153c67fa2b94a8eb1f050cc0c64e967 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:05.974137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:05.974661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:05.974751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:06.009274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:06.010596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:06.084062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1073/2175] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:06.091359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:06.093056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:06.126801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:06.178156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c48b65-8bd8-4d24-a1f3-c8c45e56165b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:06.196645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:06.197639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:06.198094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:06.198435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:06.217538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:06.217538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:06.255427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1103/2176] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:06.969817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:06.971327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:06.989612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231f8bf-c17c-48c5-ad12-3b5398213096 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:06.996481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:06.997572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:06.998009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:06.998468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] PATH_INFO: `/v3/projects/2ff97454a7e84c9d816254b1538233dd` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:07.000257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff97454a7e84c9d816254b1538233dd)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:07.002426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:07.335744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff97454a7e84c9d816254b1538233dd', 'request_id': 'req-1b11015f-9d76-4a44-9617-27d081155d69'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:07.621596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff97454a7e84c9d816254b1538233dd', 'request_id': 'req-1b11015f-9d76-4a44-9617-27d081155d69'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:07.628392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff97454a7e84c9d816254b1538233dd', 'request_id': 'req-1b11015f-9d76-4a44-9617-27d081155d69'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:07.659118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] The token cache is being invalidate because project 2ff97454a7e84c9d816254b1538233dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:07.660438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:07.667027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:07.671966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1074/2177] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:06 2022] DELETE /identity/v3/projects/2ff97454a7e84c9d816254b1538233dd => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:07.679532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:07.693191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb4bc6-b614-481c-9678-28cf21933365 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:07.763312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:07.764758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:07.764758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:07.764758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] PATH_INFO: `/v3/users/c794dc7aabad4868bc59650afafa977e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:07.765660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c794dc7aabad4868bc59650afafa977e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:07.768028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:08.001146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c794dc7aabad4868bc59650afafa977e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:08.018031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c794dc7aabad4868bc59650afafa977e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:08.026968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c794dc7aabad4868bc59650afafa977e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:08.027799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c794dc7aabad4868bc59650afafa977e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:08.104929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d93529b9-6060-47c8-9c0f-6f8fe1be334b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c794dc7aabad4868bc59650afafa977e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:08.113314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1104/2178] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:07 2022] DELETE /identity/v3/users/c794dc7aabad4868bc59650afafa977e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:08.901576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:08.903085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:08.952411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11015f-9d76-4a44-9617-27d081155d69 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:09.114755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:09.115862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:09.116147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:09.116834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] PATH_INFO: `/v3/projects/1e78d847d86e4dbebd7371658b6a099f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:09.121266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e78d847d86e4dbebd7371658b6a099f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:09.123773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:09.181517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:09.334287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e78d847d86e4dbebd7371658b6a099f', 'request_id': 'req-f680ad8c-22f2-43be-a107-126eae81e2d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:09.449660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e78d847d86e4dbebd7371658b6a099f', 'request_id': 'req-f680ad8c-22f2-43be-a107-126eae81e2d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:09.463775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e78d847d86e4dbebd7371658b6a099f', 'request_id': 'req-f680ad8c-22f2-43be-a107-126eae81e2d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:09.499456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] The token cache is being invalidate because project 1e78d847d86e4dbebd7371658b6a099f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:09.499858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:09.500644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f680ad8c-22f2-43be-a107-126eae81e2d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:09.508660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1075/2179] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:08 2022] DELETE /identity/v3/projects/1e78d847d86e4dbebd7371658b6a099f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:09.513659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:09.514175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:09.514513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:09.546301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:09.547600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:09.628877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:09.678525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1105/2180] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:09.686428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:09.686742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:09.686984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:09.721459 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:09.725235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:09.736099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1076/2181] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:09.741422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:09.762990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f98e22-c83a-4892-9934-eda740d18523 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:09.767791 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:09.768413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:09.768695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:09.768964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:09.769628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:49:09.770119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:09.777688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:09.809373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1106/2182] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:49:09.814047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:09.833289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084beab-7485-4b3c-a6ae-0b4f537988ac None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:09.856071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:09.856702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:09.856986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:09.857255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:09.857897 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:09.863102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:10.086864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1077/2183] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:09 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:10.091068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:10.112418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df20b-d30a-4d3b-b2b3-de76d855d226 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:10.400695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:10.407283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:10.407283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:10.407283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:10.407283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:10.407283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:10.590982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1107/2184] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:10 2022] POST /identity/v3/users => generated 293 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:10.599411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:10.656781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60222710-b25a-42fb-af6c-4b54a28990f7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:10.688453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:10.688453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:10.688453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:10.688453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:10.688921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:10.688921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:10.691001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1078/2185] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:10.695426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:10.718411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e014d-9d32-4e56-abd5-3c2c3677ab18 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:10.728431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:10.729877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:10.730199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:10.730474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:10.731175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:10.732893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270eb6e8-4d4e-4941-887a-5fcdd3bef1aa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:10.753438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1108/2186] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:10.762350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:10.774200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68557c9-3a93-41bb-84f7-d023bf864d2e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:10.786715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:10.787750 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:10.788228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:10.788710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] PATH_INFO: `/v3/projects/9c3c7c359ef44c70857dd623f9337da3/users/74be6a0ba62c40d0b3f826c01e486dab/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:10.841803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3c7c359ef44c70857dd623f9337da3, user_id=74be6a0ba62c40d0b3f826c01e486dab, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:10.845267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:11.479489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1079/2187] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:10 2022] PUT /identity/v3/projects/9c3c7c359ef44c70857dd623f9337da3/users/74be6a0ba62c40d0b3f826c01e486dab/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:49:11.490884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:11.491168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:11.491442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:11.525611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] MFA Rules not processed for user `74be6a0ba62c40d0b3f826c01e486dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:11.530642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:11.598515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1109/2188] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:11.616635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:11.616635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:11.677613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19af3d6-1f96-4800-b50a-f326d16ce51d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:11.678603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:11.679259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:11.679489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:11.679871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:11.690761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:11.692075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de4782-c571-4d38-84a0-ed5991925171 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:11.712057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1080/2189] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:11.763016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:11.866413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:11.921645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaab0a0-ec11-4bf4-983c-c8228893a74b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:11.934189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:11.937502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:11.938070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:11.938694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] PATH_INFO: `/v3/users/74be6a0ba62c40d0b3f826c01e486dab` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:11.941460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74be6a0ba62c40d0b3f826c01e486dab)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:11.943891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:12.250635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74be6a0ba62c40d0b3f826c01e486dab'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:12.284895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74be6a0ba62c40d0b3f826c01e486dab'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:12.300156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74be6a0ba62c40d0b3f826c01e486dab'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:12.301666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74be6a0ba62c40d0b3f826c01e486dab'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:12.472854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74be6a0ba62c40d0b3f826c01e486dab'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:12.479823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1110/2190] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:11 2022] DELETE /identity/v3/users/74be6a0ba62c40d0b3f826c01e486dab => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:12.485146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc82a-93bc-44d5-80f5-a1f8e9fd5c68 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:12.490220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc82a-93bc-44d5-80f5-a1f8e9fd5c68 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:12.490220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc82a-93bc-44d5-80f5-a1f8e9fd5c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:12.512241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bedbc82a-93bc-44d5-80f5-a1f8e9fd5c68 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:12.513487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bedbc82a-93bc-44d5-80f5-a1f8e9fd5c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:12.616488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:12.627358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1081/2191] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:12.637877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:12.640259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:12.700293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35623b27-1415-401b-a3ab-12c99ed7d98d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:12.704048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:12.704655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:12.704935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:12.705207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:12.716045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:12.717462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:12.728446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1111/2192] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:13.347710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.350986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.350986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.374610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:13.375854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:13.441490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1082/2193] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:13.456178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:13.458385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:13.482622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:13.484753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:13.505589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04d67c-86fc-4f1a-916a-4a3735246bf2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:13.512906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:13.515446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.526282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.526282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] PATH_INFO: `/v3/projects/9c3c7c359ef44c70857dd623f9337da3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.526282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3c7c359ef44c70857dd623f9337da3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:13.526282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:13.557423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:13.563409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95049073-2ef7-41fa-ae92-879a3b8124db service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:13.572345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:13.572345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.572345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.572345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.578248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:13.579850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeace0d-bd22-474d-90c3-6c717578d32a service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:13.588460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1112/2194] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:13.730573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3c7c359ef44c70857dd623f9337da3', 'request_id': 'req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:13.811627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3c7c359ef44c70857dd623f9337da3', 'request_id': 'req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:13.818423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3c7c359ef44c70857dd623f9337da3', 'request_id': 'req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:13.832375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] The token cache is being invalidate because project 9c3c7c359ef44c70857dd623f9337da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:13.832674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:13.833369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8fa6c069-91e6-4b8c-a257-4f4d61c267cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:13.834934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1083/2195] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:13 2022] DELETE /identity/v3/projects/9c3c7c359ef44c70857dd623f9337da3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:49:13.839845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.840292 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.840404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.860849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:13.862290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:13.922653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1113/2196] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:49:13.930471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.931445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.931716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.951412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:13.952605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:13.960424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1084/2197] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:13.964611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:13.974296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323623-809f-4e89-a3fd-12e64f47f068 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:13.979407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:13.980215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:13.980734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:13.981432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:13.982204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:49:13.982513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:13.984212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:13.996596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1114/2198] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:49:14.000857 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.010589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a794a3d-786e-4471-8bf2-0af35b73eff4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.015824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.016418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.016695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.016963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.017596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.019572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99550d9-2c4b-4766-95dd-94c5e38df7af admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.131704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1085/2199] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:13 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:14.140457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.151135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a9832-791a-42a4-88ca-ed4221d5944f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.167446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38653e-0643-4ea3-b13b-5ac3ce8d0281 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.168058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38653e-0643-4ea3-b13b-5ac3ce8d0281 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.168058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38653e-0643-4ea3-b13b-5ac3ce8d0281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.197029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0e38653e-0643-4ea3-b13b-5ac3ce8d0281 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:14.198281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e38653e-0643-4ea3-b13b-5ac3ce8d0281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:14.211002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.211616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.212437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.212709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.213423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.215392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.243936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:14.255814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1086/2200] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:49:14.260061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.260061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.260254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.281861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] MFA Rules not processed for user `b9a4f3995e1242de9d4bc87ce0608265`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:14.283144 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:14.331392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1115/2201] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:14.331392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.332982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:14.345629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1087/2202] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:14.346648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.365299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff6125-c833-4c69-b573-d9bf1d75d11a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.371881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.372933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.374157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.375635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.378142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.379865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.395336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1088/2203] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:14.400425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.412334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03e94b-efa3-45eb-bf01-60fbf41cf478 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.417888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.418628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.419236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.419581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.420448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.422614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.427201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6523024-41d1-4cc7-9cb1-40dd9a95a844 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.438208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1089/2204] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:49:14.439160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.442202 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.449680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.449680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.449680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.450841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404beea-b38a-4ec0-b9d6-cf938be7293e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.453700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.455766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c448c-18f5-4dc7-aa69-0ee7d59cf78d service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.457331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.458157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.458500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.458842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] PATH_INFO: `/v3/projects/f1193f380e784a91a7499fac4a151811/users/f52b1a692eb84c499ce05532d0099046/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.466928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1116/2205] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:14.476432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1193f380e784a91a7499fac4a151811, user_id=f52b1a692eb84c499ce05532d0099046, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.479479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.495307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:14.550949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1090/2206] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:14 2022] PUT /identity/v3/projects/f1193f380e784a91a7499fac4a151811/users/f52b1a692eb84c499ce05532d0099046/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:14.561926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b25f7-d0b2-4dfb-bb8a-cdb0f92469ea None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.562428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b25f7-d0b2-4dfb-bb8a-cdb0f92469ea None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.562714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b25f7-d0b2-4dfb-bb8a-cdb0f92469ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.599053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b46b25f7-d0b2-4dfb-bb8a-cdb0f92469ea None None] MFA Rules not processed for user `f52b1a692eb84c499ce05532d0099046`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:14.600909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b46b25f7-d0b2-4dfb-bb8a-cdb0f92469ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:14.674781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1117/2207] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:14.679410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.765801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.766112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.766392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.767931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1e032-1281-4560-9e44-f53a3554a4b7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.773211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.773900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.774899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.776187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.776187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.778174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:14.789047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:14.790334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:14.855558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1118/2208] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:49:14.873842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.904181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1091/2209] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/projects => generated 375 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:14.905587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:14.955455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:14.963358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbbc08-aaf4-4fb1-99f9-e7336c1aadc5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.979176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66283d69-3982-4f51-a59b-2d12acc1e447 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:14.984984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:14.985864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:14.986247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:14.986630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:14.987342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:14.989668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.007910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:15.036151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.036827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.037115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.037389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.093801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.095386 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.103310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1119/2210] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:15.123884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1092/2211] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:14 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:15.125283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.136754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e4f0-0148-4bfd-956e-2ff5fc024032 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.141596 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.142304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.142650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.142988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.143729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.145399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.162079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1120/2212] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:15.166896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.177913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c14cf-9f75-4dfb-a4f8-6d9c96168a0f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.183506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.184235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.184588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.185539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] PATH_INFO: `/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a/users/a0755306ac4c4f13b7bcb59ba6841164/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.204204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae8c612e9a4524a50d3c3ffdbaf47a, user_id=a0755306ac4c4f13b7bcb59ba6841164, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.207514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.277467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1093/2213] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:15 2022] PUT /identity/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a/users/a0755306ac4c4f13b7bcb59ba6841164/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:15.281650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.298554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728011f-243b-45ee-abb2-38c318614544 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.365640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.365640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.365640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.365640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.366273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.367051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d878-efbf-43c9-bf0c-ae631ff616e1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.378821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1121/2214] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:15.383153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.394304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66451fed-5d2f-44a3-a751-900944ed4640 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.399820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.400473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.400756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.401029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] PATH_INFO: `/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a/users/a0755306ac4c4f13b7bcb59ba6841164/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.402978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae8c612e9a4524a50d3c3ffdbaf47a, user_id=a0755306ac4c4f13b7bcb59ba6841164, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.405787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.416922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1094/2215] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:15 2022] PUT /identity/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a/users/a0755306ac4c4f13b7bcb59ba6841164/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:15.487973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517228-891c-4189-85a6-0ed0a36d28c1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.488325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517228-891c-4189-85a6-0ed0a36d28c1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.488532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517228-891c-4189-85a6-0ed0a36d28c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.521265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0c517228-891c-4189-85a6-0ed0a36d28c1 None None] MFA Rules not processed for user `a0755306ac4c4f13b7bcb59ba6841164`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:15.521538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0c517228-891c-4189-85a6-0ed0a36d28c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:15.579878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:15.601306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1122/2216] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:15.612745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.614595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:15.675959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbca368-8b25-4dbc-9981-a9ef8c6de903 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.680933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.681633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.681914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.682216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.740145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:15.741588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:15.750733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1095/2217] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:15.831662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd833b-e24b-4303-a3ba-34aa88252323 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.831955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd833b-e24b-4303-a3ba-34aa88252323 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:15.832233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd833b-e24b-4303-a3ba-34aa88252323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:15.852098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-50bd833b-e24b-4303-a3ba-34aa88252323 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:15.853872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-50bd833b-e24b-4303-a3ba-34aa88252323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:15.914749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1123/2218] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:49:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:15.926910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:15.973119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:15.993872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5b3ca-477c-4e2b-a1d6-8a70801a7b2c service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:15.998961 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:15.999615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:15.999893 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:16.000164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:16.009302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:16.011124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:16.019708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1096/2219] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:16.733678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:16.734020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:16.734831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:16.755242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:16.756485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:16.764052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1124/2220] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:16.778664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:16.790177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a4d3b-c51d-4bfa-9640-a2a2207a9dfc service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:16.795753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:16.796468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:16.796754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:16.797029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:16.806241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:16.807830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:16.816165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1097/2221] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:17.950684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:17.952947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:18.023121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe944-94d0-4d37-9837-2f709a24254f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:18.027892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:18.028522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:18.028801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:18.029076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] PATH_INFO: `/v3/users/60d10ce14ee3486f8e714dcc912d1527` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:18.030249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d10ce14ee3486f8e714dcc912d1527)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:18.032173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:18.228082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d10ce14ee3486f8e714dcc912d1527'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.248620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d10ce14ee3486f8e714dcc912d1527'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.262126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d10ce14ee3486f8e714dcc912d1527'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.263414 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d10ce14ee3486f8e714dcc912d1527'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.329716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f682d4bc-1dc3-4d68-805f-ceab41d068b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d10ce14ee3486f8e714dcc912d1527'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.342495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1125/2222] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:17 2022] DELETE /identity/v3/users/60d10ce14ee3486f8e714dcc912d1527 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:18.661208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:18.680998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad84ad-73ee-42c0-b27e-427f1ada006e service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:18.770367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:18.771083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:18.771427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:18.771765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] PATH_INFO: `/v3/projects/c538a572650b4574b9f6f9c8853e9a4f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:18.772892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c538a572650b4574b9f6f9c8853e9a4f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:18.775490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:18.900875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c538a572650b4574b9f6f9c8853e9a4f', 'request_id': 'req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.980200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c538a572650b4574b9f6f9c8853e9a4f', 'request_id': 'req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.986795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c538a572650b4574b9f6f9c8853e9a4f', 'request_id': 'req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:18.999679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] The token cache is being invalidate because project c538a572650b4574b9f6f9c8853e9a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:18.999971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:19.000634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a26e489-d27f-4689-96d7-a1c43e3f8a5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:19.008712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1098/2223] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:18 2022] DELETE /identity/v3/projects/c538a572650b4574b9f6f9c8853e9a4f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:19.011635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bb0c-f483-4be2-b9a6-56b5e681d126 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.012137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bb0c-f483-4be2-b9a6-56b5e681d126 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.012243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bb0c-f483-4be2-b9a6-56b5e681d126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.023450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:19.036899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ee01bb0c-f483-4be2-b9a6-56b5e681d126 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:19.038109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ee01bb0c-f483-4be2-b9a6-56b5e681d126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:19.048790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.049081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.049381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.071505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:19.071505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:19.100054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1126/2224] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:19.107649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.107938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.108213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.119294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:19.128809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:19.130038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:19.132569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1099/2225] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:19.141092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1127/2226] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:49:19.145406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.147750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.149665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:19.157791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53b5b-fc71-48ba-b371-bce285ff06ca None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.164120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.164749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.164749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.164945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.165603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:49:19.165911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.167597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.185943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1100/2227] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:49:19.199483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.222550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff314ef-75cf-415a-88bd-1f978a7058a2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.259325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.259325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.259325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.259325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.260176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.260284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31291d1-33b1-4485-bb73-edf7fc4fa2bd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.260386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.260481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.260481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.260481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.260481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.268180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.270387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.281109 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1128/2228] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:49:19.378607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1101/2229] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:19.382755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.394368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85d370-fac3-41bd-890a-4906d6d5d628 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.458332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.458919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.459049 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.459243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.459903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.463922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.615368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1129/2230] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/users => generated 284 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:19.616454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.629816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8afff-7477-4ac6-ac33-d7191908b788 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.641813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.642806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.643240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.644433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.645458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.650108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.673635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1102/2231] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:49:19.678603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.692561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f6cf2-72d7-452d-8849-815afb1fde34 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.704676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.706385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.706385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.707503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.709020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.711907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.749342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1130/2232] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:19.750684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.761772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d356256-eae1-4931-b3e5-0c1440f532bf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.767222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.768016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.768298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.768743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] PATH_INFO: `/v3/projects/440b1b0a705e49408770f45d801ab3da/users/c10262cd091149c792105230127b44f8/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.797647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440b1b0a705e49408770f45d801ab3da, user_id=c10262cd091149c792105230127b44f8, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.800576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2205342-001b-43f1-b7c0-fb0360919c36 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.813559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:19.815498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:19.833201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedc521-04cc-40bd-a639-cc782ef93673 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:19.843448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:19.844771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.845036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.845470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] PATH_INFO: `/v3/users/1c6f49dbafbc41fc989d4d12acc0a28c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.848796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6f49dbafbc41fc989d4d12acc0a28c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:19.856569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:19.860627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:19.901289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1103/2233] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:19 2022] PUT /identity/v3/projects/440b1b0a705e49408770f45d801ab3da/users/c10262cd091149c792105230127b44f8/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:19.916399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:19.916735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:19.916959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:19.965524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] MFA Rules not processed for user `c10262cd091149c792105230127b44f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:19.967456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.063299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1104/2234] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:49:20.075991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:20.077469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.143169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6f49dbafbc41fc989d4d12acc0a28c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.143739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09277be-f53f-4b95-bd4c-6cbd7f990c33 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:20.147541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:20.148245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.148734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.148896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.161518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6f49dbafbc41fc989d4d12acc0a28c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.164867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:20.166392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd05d4-ee8b-4a1e-ab79-491e2c8fa699 service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:20.173019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6f49dbafbc41fc989d4d12acc0a28c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.173965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6f49dbafbc41fc989d4d12acc0a28c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.183667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1105/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:49:20.252634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6f49dbafbc41fc989d4d12acc0a28c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.262958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1131/2236] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:19 2022] DELETE /identity/v3/users/1c6f49dbafbc41fc989d4d12acc0a28c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:49:20.267577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.267937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.268343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.315794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:20.317716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.337883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:20.339689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.359265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:20.437650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1106/2237] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:20.453217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:20.456373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.496897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac21e-572a-4afd-88a6-2d002c7c5923 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:20.513293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:20.514090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.514267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.514548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] PATH_INFO: `/v3/users/f52b1a692eb84c499ce05532d0099046` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.515964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52b1a692eb84c499ce05532d0099046)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:20.517898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:20.535623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f464e-7f5b-428a-b767-4fb7c2467463 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:20.541567 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:20.542703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.543409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.554279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.570319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:20.573071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:20.584796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1107/2238] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:49:20.753266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52b1a692eb84c499ce05532d0099046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.771523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52b1a692eb84c499ce05532d0099046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.781448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52b1a692eb84c499ce05532d0099046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.782289 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52b1a692eb84c499ce05532d0099046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.878021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f807b1bd-ac4f-4fdd-b53f-baca952c9049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52b1a692eb84c499ce05532d0099046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:20.884232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:20.884674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:20.885497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.887235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1132/2239] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:20 2022] DELETE /identity/v3/users/f52b1a692eb84c499ce05532d0099046 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:20.891610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.892131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.892694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.905201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcd780-cbc9-4474-9442-627cd61bc4ae service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:20.914882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:20.923465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:20.977053 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:20.977692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:20.977974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:20.978278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] PATH_INFO: `/v3/projects/aba6e95f376843d29b7fee2ef2b1b869` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:20.979481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba6e95f376843d29b7fee2ef2b1b869)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:20.982085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:21.017702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1133/2240] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:21.031155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:21.032746 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:21.095858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7318d-fbd2-43aa-93de-9e7b3aae92ce None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:21.099811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:21.100475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:21.100761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:21.101079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:21.120982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:21.122845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:21.147026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba6e95f376843d29b7fee2ef2b1b869', 'request_id': 'req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.147246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1134/2241] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:21.223909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba6e95f376843d29b7fee2ef2b1b869', 'request_id': 'req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.230372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:21.247472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba6e95f376843d29b7fee2ef2b1b869', 'request_id': 'req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.276871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] The token cache is being invalidate because project aba6e95f376843d29b7fee2ef2b1b869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:21.276871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.278273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.280334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1108/2242] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:20 2022] DELETE /identity/v3/projects/aba6e95f376843d29b7fee2ef2b1b869 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:21.752048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:21.753265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:21.767222 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e887-d7b5-4bc7-91c5-0c104144b653 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:21.831443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:21.832156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:21.832597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:21.833060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] PATH_INFO: `/v3/projects/f1193f380e784a91a7499fac4a151811` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:21.834866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1193f380e784a91a7499fac4a151811)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:21.838944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:21.952003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:21.956278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1193f380e784a91a7499fac4a151811', 'request_id': 'req-d051cf32-755e-4159-9ebe-9c43244133c1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.963807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1193f380e784a91a7499fac4a151811', 'request_id': 'req-d051cf32-755e-4159-9ebe-9c43244133c1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:21.969477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1193f380e784a91a7499fac4a151811', 'request_id': 'req-d051cf32-755e-4159-9ebe-9c43244133c1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.077518 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] The token cache is being invalidate because project f1193f380e784a91a7499fac4a151811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:22.077846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.078504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.080103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1135/2243] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:21 2022] DELETE /identity/v3/projects/f1193f380e784a91a7499fac4a151811 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:22.084610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:22.087831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:22.103611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86783ccb-a9ac-4306-99f0-8b8d91d8cc03 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:22.166937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:22.167607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:22.167886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:22.168186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] PATH_INFO: `/v3/users/a0755306ac4c4f13b7bcb59ba6841164` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:22.169708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0755306ac4c4f13b7bcb59ba6841164)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:22.171828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:22.386162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0755306ac4c4f13b7bcb59ba6841164'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.413449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0755306ac4c4f13b7bcb59ba6841164'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.424581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0755306ac4c4f13b7bcb59ba6841164'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.570288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0755306ac4c4f13b7bcb59ba6841164'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.573899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-466e71a2-b96e-452d-9de2-09e1b9de644f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0755306ac4c4f13b7bcb59ba6841164'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:22.591225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1109/2244] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:22 2022] DELETE /identity/v3/users/a0755306ac4c4f13b7bcb59ba6841164 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:23.093492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:23.094919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:23.108476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051cf32-755e-4159-9ebe-9c43244133c1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:23.176159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:23.176812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.177087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.177355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] PATH_INFO: `/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.178513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cae8c612e9a4524a50d3c3ffdbaf47a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:23.180427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:23.301835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cae8c612e9a4524a50d3c3ffdbaf47a', 'request_id': 'req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:23.310976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cae8c612e9a4524a50d3c3ffdbaf47a', 'request_id': 'req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:23.317771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cae8c612e9a4524a50d3c3ffdbaf47a', 'request_id': 'req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:23.408125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] The token cache is being invalidate because project 0cae8c612e9a4524a50d3c3ffdbaf47a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:23.408418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:23.409279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-aeec5fcc-95e3-493e-b7c9-2d854d3cc8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:23.412590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1136/2245] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:23 2022] DELETE /identity/v3/projects/0cae8c612e9a4524a50d3c3ffdbaf47a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:23.418973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.419495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.419596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.451470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:23.453081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:23.534902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1110/2246] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:23.542644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.542931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.543217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.566110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:23.568351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:23.577411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1137/2247] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:23.581743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:23.600372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5c072-63d6-44ab-99bb-9a04896ce598 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:23.605832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:23.606569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.606842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.607112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.607779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:49:23.608073 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:23.609695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:23.617238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:23.623966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1111/2248] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:49:23.627508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:23.637972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7da14-999d-4d3a-b2e7-02b452bb3ea1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:23.658165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:23.658886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.659249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.659590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.660262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:23.664540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:23.753171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1138/2249] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:23 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:23.757507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:23.768717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52adcb-9c59-41c3-bd95-fdc3831e209b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:23.831755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:23.832381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.832720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.832992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.833769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:23.835752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:23.952608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1112/2250] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:23 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:23.958045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:23.971491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e7191-87f3-4d9c-a9d5-7d69c7beb61f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:23.977538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:23.978199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:23.978478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:23.978755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:23.979417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:23.981015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:23.992777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1139/2251] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:23.997044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.007971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05fd27-4a7e-4f16-bb29-3a550f1370fc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.013069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.013752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.014140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.014457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.015129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.016779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64308380-416f-4f67-8422-f5a832e4c4c6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.028283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1113/2252] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:24.032483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.043201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757a0d-de4c-48f5-8e72-8280ddd042b2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.048276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.049054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.049457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.049734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] PATH_INFO: `/v3/projects/8ab4c675bc4e41179c2de9ee9b604126/users/09ecb50a0f214c618c3a19f88a0b445a/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.066551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab4c675bc4e41179c2de9ee9b604126, user_id=09ecb50a0f214c618c3a19f88a0b445a, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.069362 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.133724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:24.140355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1140/2253] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:24 2022] PUT /identity/v3/projects/8ab4c675bc4e41179c2de9ee9b604126/users/09ecb50a0f214c618c3a19f88a0b445a/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:24.151175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.151447 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.151723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.190196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] MFA Rules not processed for user `09ecb50a0f214c618c3a19f88a0b445a`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:24.191570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:24.264028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1114/2254] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:24.268861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.343996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a40eb-e387-49cb-8f0f-edd9b503c2c2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.349033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.349770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.350083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.350406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.351115 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.353654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.459538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1141/2255] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:24 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:24.463499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.474648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ec05-04e7-4d30-b68a-d232227f0f73 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.537847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.538465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.538742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.539081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.540239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.542190 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.553891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:24.647830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1115/2256] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:24 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:24.653030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.663878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26fff6-ab2f-46eb-b3ae-34dc88de2e4e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.670390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.671158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.671453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.671727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.672384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.674015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.685570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1142/2257] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:24.689859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.700516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdef04-ec1a-4d34-8199-d7e7b870925a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.705284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.705956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.706306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.706613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.707340 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.709049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7c602-95a1-424b-bcd0-cef11ad92a1f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.721991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1116/2258] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:24.727192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:24.738193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da813b40-3b1c-4a77-91bd-42c3cae87869 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:24.743094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:24.743766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.744056 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.744458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] PATH_INFO: `/v3/projects/950f014d27574398a8251ac71f52b77a/users/13547adea8ba4da2bf95ca457c17a385/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.760642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] RBAC: Authorizing `identity:create_grant(project_id=950f014d27574398a8251ac71f52b77a, user_id=13547adea8ba4da2bf95ca457c17a385, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:24.763422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:24.871045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1143/2259] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:24 2022] PUT /identity/v3/projects/950f014d27574398a8251ac71f52b77a/users/13547adea8ba4da2bf95ca457c17a385/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:24.884196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:24.884582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:24.884933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:24.928194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] MFA Rules not processed for user `13547adea8ba4da2bf95ca457c17a385`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:24.929614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:25.009730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1117/2260] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:25.014480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.081838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15df2-e6ff-45dd-b9d3-452bcb54970c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.087239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.087842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.088116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.088384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.089032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.091684 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.207267 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1144/2261] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:49:25 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:49:25.212013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.223329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc6f70-1545-434a-97b9-4a5b580bdb88 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.290373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.294906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.294906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.294906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.294906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.294906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.410188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1118/2262] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:49:25 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:49:25.414501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.425218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e78e-7728-4384-b5d0-bb021b5f1921 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.430416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.431107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.431451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.431790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.432513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.434216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.446068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1145/2263] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:25.450575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.461120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa629c-907b-4e31-a5c2-7153942c7a01 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.465728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.466450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.466770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.467094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] PATH_INFO: `/v3/projects/edebe2774f3347909c43ba3a7301f0cc/users/c813baef72a34ba8b19063a8ebc280d4/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.483183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edebe2774f3347909c43ba3a7301f0cc, user_id=c813baef72a34ba8b19063a8ebc280d4, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.486617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.566544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1119/2264] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:25 2022] PUT /identity/v3/projects/edebe2774f3347909c43ba3a7301f0cc/users/c813baef72a34ba8b19063a8ebc280d4/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:25.571274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.586952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c360e-d160-4df7-9544-46f326246bd5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.610902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:25.660527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.661573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.662014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.662527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.663179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.665440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739f18d-c555-41e9-8dd9-0716fd26e2b4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.678113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1146/2265] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:49:25.683204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.694798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5453-f9a5-44cc-9108-afa2ab91edc3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.703892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.705066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.706089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.706089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] PATH_INFO: `/v3/projects/edebe2774f3347909c43ba3a7301f0cc/users/c813baef72a34ba8b19063a8ebc280d4/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.712885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edebe2774f3347909c43ba3a7301f0cc, user_id=c813baef72a34ba8b19063a8ebc280d4, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:25.716236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:25.729590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1120/2266] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:49:25 2022] PUT /identity/v3/projects/edebe2774f3347909c43ba3a7301f0cc/users/c813baef72a34ba8b19063a8ebc280d4/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:25.740349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.740785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.741157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:25.772305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] MFA Rules not processed for user `c813baef72a34ba8b19063a8ebc280d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:25.774587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:25.869316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1147/2267] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:25.886549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:25.888367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:25.936794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:25.977849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bbdcb-0b40-44cd-9a5d-0b3a664934c9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:25.982087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:25.982972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:25.983302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:25.983662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:26.078898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:26.080579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:26.096740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1121/2268] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:49:30.549141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:30.559698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af217534-55a8-441a-ad02-28a4000722d8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:30.563532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:30.564156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:30.564421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:30.564692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:30.633755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:30.638011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:30.649208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1148/2269] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:32.333624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:32.444131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358117-cac9-4dcf-a690-1d199fc7def5 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:32.449364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:32.449908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:32.450222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:32.450492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:32.460841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:32.462285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf021-28af-4423-8086-218a4404f2bf service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:32.472361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1122/2270] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:40.422887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:40.466911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:40.484973 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb10378-0da1-4d50-ae8e-f93b9aac7cac service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:40.489571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:40.490159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:40.490440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:40.490756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] PATH_INFO: `/v3/users/09ecb50a0f214c618c3a19f88a0b445a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:40.491882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ecb50a0f214c618c3a19f88a0b445a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:40.493778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:40.815480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ecb50a0f214c618c3a19f88a0b445a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:40.827385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ecb50a0f214c618c3a19f88a0b445a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:40.834293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ecb50a0f214c618c3a19f88a0b445a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:40.834968 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ecb50a0f214c618c3a19f88a0b445a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:40.904157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ecb50a0f214c618c3a19f88a0b445a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:40.910591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1149/2271] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:40 2022] DELETE /identity/v3/users/09ecb50a0f214c618c3a19f88a0b445a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:40.915142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:40.915641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:40.915920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:40.935544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:49:40.936785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:40.999126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1123/2272] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:49:41.010513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:41.011144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:41.071365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e1d28-b583-4487-abe2-6bdff3598944 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:41.075072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:41.075681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:41.075962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:41.076236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:41.086824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:41.088334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:41.176803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1150/2273] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:49:41.719307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:41.723617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:41.737188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5fbf2-9d82-4a74-8586-fa9092c40f4d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:41.750709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:41.752040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:41.752501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:41.752970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] PATH_INFO: `/v3/projects/8ab4c675bc4e41179c2de9ee9b604126` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:41.757087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab4c675bc4e41179c2de9ee9b604126)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:41.758915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:41.925448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab4c675bc4e41179c2de9ee9b604126', 'request_id': 'req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.021557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab4c675bc4e41179c2de9ee9b604126', 'request_id': 'req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.026917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:42.028234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab4c675bc4e41179c2de9ee9b604126', 'request_id': 'req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.039402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] The token cache is being invalidate because project 8ab4c675bc4e41179c2de9ee9b604126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:42.039691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.040327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.041874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1124/2274] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:41 2022] DELETE /identity/v3/projects/8ab4c675bc4e41179c2de9ee9b604126 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:42.046166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:42.056447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1674e4-e4d7-4ac3-b345-48aaf4005a5f service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:42.114934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:42.114934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:42.114934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:42.114934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] PATH_INFO: `/v3/users/13547adea8ba4da2bf95ca457c17a385` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:42.116598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13547adea8ba4da2bf95ca457c17a385)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:42.118698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:42.126983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:42.350775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13547adea8ba4da2bf95ca457c17a385'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.362956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13547adea8ba4da2bf95ca457c17a385'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.370659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13547adea8ba4da2bf95ca457c17a385'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.371371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13547adea8ba4da2bf95ca457c17a385'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.478551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13547adea8ba4da2bf95ca457c17a385'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:42.484920 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1151/2275] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:42 2022] DELETE /identity/v3/users/13547adea8ba4da2bf95ca457c17a385 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:42.911071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:42.913017 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:42.926938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a202f-d6b6-4392-9e4d-a79ed1dcf5b0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:42.994707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:42.995348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:42.995631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:42.995902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] PATH_INFO: `/v3/projects/950f014d27574398a8251ac71f52b77a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:42.997117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=950f014d27574398a8251ac71f52b77a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:42.999096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:43.117623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950f014d27574398a8251ac71f52b77a', 'request_id': 'req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.189454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950f014d27574398a8251ac71f52b77a', 'request_id': 'req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.195616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950f014d27574398a8251ac71f52b77a', 'request_id': 'req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.207549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] The token cache is being invalidate because project 950f014d27574398a8251ac71f52b77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:43.211122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.211122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.211122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1125/2276] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:42 2022] DELETE /identity/v3/projects/950f014d27574398a8251ac71f52b77a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:43.213998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:43.215468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:43.228640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c76bf7-b1ea-4d22-ba8b-1d730e43f333 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:43.284371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:43.285000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:43.285282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:43.285564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] PATH_INFO: `/v3/users/c813baef72a34ba8b19063a8ebc280d4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:43.286711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c813baef72a34ba8b19063a8ebc280d4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:43.288679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:43.541058 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c813baef72a34ba8b19063a8ebc280d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.554029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c813baef72a34ba8b19063a8ebc280d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.561128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c813baef72a34ba8b19063a8ebc280d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.561806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c813baef72a34ba8b19063a8ebc280d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.638498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2c4c6948-d624-431d-bb9a-64bc4c540121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c813baef72a34ba8b19063a8ebc280d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:43.644693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1152/2277] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:49:43 2022] DELETE /identity/v3/users/c813baef72a34ba8b19063a8ebc280d4 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:44.178473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:49:44.178473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:49:44.191913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ebc1a-cdc0-49c7-9d25-7fbbc548b65d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:49:44.256341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:49:44.256981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:49:44.257266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:49:44.257538 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] PATH_INFO: `/v3/projects/edebe2774f3347909c43ba3a7301f0cc` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:49:44.258713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edebe2774f3347909c43ba3a7301f0cc)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:49:44.260702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:49:44.288065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:44.559659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edebe2774f3347909c43ba3a7301f0cc', 'request_id': 'req-a1411368-c032-410c-b0f7-ce995a6a1a33'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:44.627043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edebe2774f3347909c43ba3a7301f0cc', 'request_id': 'req-a1411368-c032-410c-b0f7-ce995a6a1a33'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:44.636630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edebe2774f3347909c43ba3a7301f0cc', 'request_id': 'req-a1411368-c032-410c-b0f7-ce995a6a1a33'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:44.648904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] The token cache is being invalidate because project edebe2774f3347909c43ba3a7301f0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:49:44.649421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:44.650556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:49:44.652298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1126/2278] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:49:44 2022] DELETE /identity/v3/projects/edebe2774f3347909c43ba3a7301f0cc => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:49:53.641127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=77,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:49:54.650659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:09.841979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:09.842511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:09.842624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:09.862764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:09.864005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:09.920295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1153/2279] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:50:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:09.945465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:09.947429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:10.017852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1411368-c032-410c-b0f7-ce995a6a1a33 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:10.023518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:10.024121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:10.024404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:10.024679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:10.032662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:10.034281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:10.042626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1127/2280] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:50:17.305996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:17.307489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:17.372581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cab20-184a-49ee-a440-151327d4c0dc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:17.376820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:17.377516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:17.378040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:17.378402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:17.380835 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:17.453576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:17.455172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a240a-e4fa-48fc-acc1-a503ff7c55a5 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:17.470271 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1154/2281] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:20.042328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=32 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:22.222039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:22.223563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:22.291835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b059b1-60a4-4d49-a438-aae846949b2e service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:22.298362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:22.298362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:22.298362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:22.298362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] PATH_INFO: `/v3/users/c10262cd091149c792105230127b44f8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:22.298765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10262cd091149c792105230127b44f8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:22.301057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:22.559330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10262cd091149c792105230127b44f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:22.572235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10262cd091149c792105230127b44f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:22.579091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10262cd091149c792105230127b44f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:22.669799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10262cd091149c792105230127b44f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:22.670431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10262cd091149c792105230127b44f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:22.677532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1128/2282] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:50:22 2022] DELETE /identity/v3/users/c10262cd091149c792105230127b44f8 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:50:22.682089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:22.682599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:22.682719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:22.701906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:22.703186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:22.755403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:22.766897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1155/2283] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:22.776485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:22.777879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:22.837374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72204de5-4db1-4dee-93e3-d356e52df7dc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:22.841050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:22.841644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:22.841918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:22.842220 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:22.852058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:22.853482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29005655-3767-4de0-94f1-5fb20fd72dab service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:22.863877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1129/2284] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:23.522711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:23.524160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:23.537631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361d7cd-55a8-4d5a-876d-3d38e0818b51 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:23.542866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:23.543650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:23.543946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:23.544268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] PATH_INFO: `/v3/projects/440b1b0a705e49408770f45d801ab3da` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:23.545440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=440b1b0a705e49408770f45d801ab3da)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:23.547479 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:23.700259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440b1b0a705e49408770f45d801ab3da', 'request_id': 'req-07d55880-2d63-41bd-b4b2-3d2f36079b4e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:23.707748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440b1b0a705e49408770f45d801ab3da', 'request_id': 'req-07d55880-2d63-41bd-b4b2-3d2f36079b4e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:23.714337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440b1b0a705e49408770f45d801ab3da', 'request_id': 'req-07d55880-2d63-41bd-b4b2-3d2f36079b4e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:23.846745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] The token cache is being invalidate because project 440b1b0a705e49408770f45d801ab3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:50:23.857606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:23.857606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-07d55880-2d63-41bd-b4b2-3d2f36079b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:50:23.857606 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1156/2285] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:50:23 2022] DELETE /identity/v3/projects/440b1b0a705e49408770f45d801ab3da => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:50:23.857924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:23.857924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:23.857924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:23.877044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:23.879645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:23.898889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:23.973064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1130/2286] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:23.979170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:23.979461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:23.979745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.005345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:24.006622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:24.017158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1157/2287] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:24.021853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.031874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a2eef-bad3-4c45-b8c7-b356a4864688 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.038322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.040355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.040733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.041076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.042082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:50:24.042238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.044019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.061671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1131/2288] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:50:24.069827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.073365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bd536-c905-4e15-9929-a99e871cd6fd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.078627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.079236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.079515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.079787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.080460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.082425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.177306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1158/2289] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:50:24 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:50:24.181497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.192050 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428c3f-e12e-48a4-a608-55335209deed admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.249134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.249745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.250023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.250343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.250973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.252922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.385615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1132/2290] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:50:24 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:50:24.386983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.398664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef55a-8fef-4d27-83c0-5e2d682453c8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.403935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.404704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.404976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.405618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.405978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.407648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.423183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1159/2291] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:50:24.425892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.436920 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82022b-6b15-4bcd-9f02-d750662ec767 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.442980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.443589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.443865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.444755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.445629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.447698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6aa3b-5cb9-4d0b-ade3-d5fb7d2eb178 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.459952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1133/2292] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:50:24.464103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.474975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaddb9-19d4-439e-abeb-70f62e70e9d8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.479603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.480252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.487254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.487254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] PATH_INFO: `/v3/projects/3e08177037d74900a886e0aa2ae057ae/users/1c081d9bf3744177a2d2d50962a38027/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.496601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e08177037d74900a886e0aa2ae057ae, user_id=1c081d9bf3744177a2d2d50962a38027, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.499389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.570007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1160/2293] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:50:24 2022] PUT /identity/v3/projects/3e08177037d74900a886e0aa2ae057ae/users/1c081d9bf3744177a2d2d50962a38027/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:50:24.574840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ea01-c26a-46a2-a8dd-ea95380d1344 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.575182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ea01-c26a-46a2-a8dd-ea95380d1344 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.575467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ea01-c26a-46a2-a8dd-ea95380d1344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.595468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2bd9ea01-c26a-46a2-a8dd-ea95380d1344 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:24.597368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2bd9ea01-c26a-46a2-a8dd-ea95380d1344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:24.659317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1134/2294] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:24.669079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:24.670807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:24.730762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13619ebf-ca31-4ef5-a6d8-e79dc584b92f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:24.734361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:24.734962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:24.735237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:24.735506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:24.744205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:24.745610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df59ff-c9a5-46b6-909d-c183ff3d60c4 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:24.754845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1161/2295] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:28.841702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:28.841702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:28.841702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:28.841702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1135/2296] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:50:28.844273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864876f-29a8-4c30-b952-d8ad83ea53fb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:28.844699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864876f-29a8-4c30-b952-d8ad83ea53fb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:28.844825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864876f-29a8-4c30-b952-d8ad83ea53fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:28.866232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0864876f-29a8-4c30-b952-d8ad83ea53fb None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:28.867744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0864876f-29a8-4c30-b952-d8ad83ea53fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:28.894729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:28.974610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1162/2297] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:50:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:28.987871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:28.989735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:29.028781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:29.062674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27faeaa-d3cd-4b56-82cb-162d4877d5bf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:29.067485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:29.068236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.068617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.069018 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.083511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:29.086130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309daa-e48c-4970-bf1c-3042be420380 service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:29.102288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1136/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:29.316236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.324017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.324017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.324017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1163/2299] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:50:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:50:29.324952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8831d0-cc73-409b-936d-1e564c0c19c1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.325239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8831d0-cc73-409b-936d-1e564c0c19c1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.325512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8831d0-cc73-409b-936d-1e564c0c19c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.347304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fa8831d0-cc73-409b-936d-1e564c0c19c1 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:29.349165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fa8831d0-cc73-409b-936d-1e564c0c19c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:29.366753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1137/2300] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:50:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:29.372685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:29.378796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:29.393370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e71eb-8986-4ff6-aed3-ac0ff2063245 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:29.396954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:29.397549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.397827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.398129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.405242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:29.406714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3cf6f-99b7-4c20-9936-20f108d305fe service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:29.415598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1164/2301] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:29.501515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33888-5ef3-4ef5-9d59-ef9ac6ac3e26 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.501926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33888-5ef3-4ef5-9d59-ef9ac6ac3e26 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.502333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d33888-5ef3-4ef5-9d59-ef9ac6ac3e26 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.503811 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1138/2302] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:50:29.509444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e488-8610-42f8-82ae-858e56dfcf38 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.509727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e488-8610-42f8-82ae-858e56dfcf38 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.510006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043e488-8610-42f8-82ae-858e56dfcf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.532628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8043e488-8610-42f8-82ae-858e56dfcf38 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:29.534604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8043e488-8610-42f8-82ae-858e56dfcf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:29.545417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1165/2303] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:50:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:50:29.557885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d33888-5ef3-4ef5-9d59-ef9ac6ac3e26 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:29.570788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d33888-5ef3-4ef5-9d59-ef9ac6ac3e26 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:29.587384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:29.588114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:29.588460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:29.588743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:29.604230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:29.605886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:29.635659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1139/2304] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:30.728936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:30.729436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:30.729544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:30.763912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] MFA Rules not processed for user `1c081d9bf3744177a2d2d50962a38027`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:30.765517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:30.847378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1166/2305] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:30.851380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:30.864572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b8b3-f920-405f-8410-13675e258291 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:30.870626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:30.871559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:30.871718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:30.871994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:30.872698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:30.875172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:30.985143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1140/2306] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:50:30 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:50:30.985246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:30.988460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fb5fe-140b-4b7c-8744-19f6ae026b51 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:31.051111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:31.051850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:31.052132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:31.052482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:31.053028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:31.055053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:31.192886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1167/2307] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:50:30 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:50:31.196835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:31.207032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b838d-a88b-4650-93dd-1110573c5ddc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:31.219155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:31.219810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:31.220086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:31.220312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:31.221023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:31.222708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:31.234555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1141/2308] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:50:31.238611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:31.249881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949417e5-7a06-40d4-a387-6842994fd87f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:31.255342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:31.256052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:31.256338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:31.256617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] PATH_INFO: `/v3/projects/0d80463eaafe40c7b583921eb79b4d31/users/c03f54407e3c456cb42c55bc67dd61cf/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:31.259999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:31.275928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d80463eaafe40c7b583921eb79b4d31, user_id=c03f54407e3c456cb42c55bc67dd61cf, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:31.278813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:31.347831 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1168/2309] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:50:31 2022] PUT /identity/v3/projects/0d80463eaafe40c7b583921eb79b4d31/users/c03f54407e3c456cb42c55bc67dd61cf/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:50:31.351947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:31.363325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535866b7-ef30-4008-8105-c1f720fc7e90 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:31.422915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:31.424117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:31.424394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:31.424663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:31.425303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:31.426940 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18ba8-288c-4a39-997b-cf217615480e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:31.436609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1142/2310] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:50:31.440958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:31.451407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1635e-c062-4452-8d10-311b58a50d96 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:31.456000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:31.456659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:31.457127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:31.457444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] PATH_INFO: `/v3/projects/0d80463eaafe40c7b583921eb79b4d31/users/c03f54407e3c456cb42c55bc67dd61cf/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:31.459828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d80463eaafe40c7b583921eb79b4d31, user_id=c03f54407e3c456cb42c55bc67dd61cf, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:31.462655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:31.473322 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1169/2311] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:50:31 2022] PUT /identity/v3/projects/0d80463eaafe40c7b583921eb79b4d31/users/c03f54407e3c456cb42c55bc67dd61cf/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:50:34.708803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:34.709317 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:34.709700 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:34.748679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:34.750251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:34.823150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1143/2312] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:34.841017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:34.842595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:34.936879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e834e-829b-4994-8b63-2eed7d48a8ca admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:34.943588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:34.944467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:34.944918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:34.945329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:34.954921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:34.956866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ba42b-5c57-4eb4-b120-c26261be4820 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:34.983842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1170/2313] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:50:37.779626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:37.833675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:37.895761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd97901-e15d-441c-b67b-6b5eada5df68 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:37.901253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:37.901954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:37.902274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:37.904230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:37.906227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:37.998480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:38.000240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:38.014230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1144/2314] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:38.542048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:38.542664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:38.542867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:38.577481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] MFA Rules not processed for user `c03f54407e3c456cb42c55bc67dd61cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:50:38.578767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:38.685797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1171/2315] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:50:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:50:38.694476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:38.696036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:38.716049 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ee0a-3c07-43d9-af40-7b11c549f270 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:38.732140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:38.733177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:38.733649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:38.734162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:38.835749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:38.838611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:38.850418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1145/2316] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:40.491278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:40.507096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcc30-4013-419a-ad43-eb6db7ac7211 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:40.512651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:40.513270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:40.513677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:40.513958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:40.516060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:40.559018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:40.560564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:40.637488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1172/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:50:48.844431 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=73 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:50.576302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=83,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:50:54.969389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:50:54.970913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:54.986483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20f49-6afe-48a1-b442-da657adc2a0e service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:50:54.993189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:50:54.993810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:50:54.994120 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:50:54.994396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:50:54.996152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:50:55.068693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:50:55.070392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a383f80-dc77-42eb-b691-008c7301f041 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:50:55.079631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1146/2318] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:50:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:01.432282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:01.441561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763cad-8d3f-408c-b55b-b53b5d961f90 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:01.445636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:01.446269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:01.446550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:01.446820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:01.448623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:01.458569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:01.459981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:01.467972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1173/2319] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:05.075464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=28 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:06.300402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:06.300996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:06.300996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:06.319766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] MFA Rules not processed for user `b193d0924fb448bfac2027034cb8dfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:06.328293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:06.380836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1147/2320] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:51:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:51:06.697030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:06.706127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447d0c-b46e-40e6-afd2-5c3cbf879488 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:06.709778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:06.710397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:06.710678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:06.710955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:06.712655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:06.722104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:06.723498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a0f7b-4006-43d3-beab-a24dc40a2302 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:06.731390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1174/2321] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:07.148219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:07.149640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:07.226843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a275477-d63d-4eb7-b871-27d9dfb92b8d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:07.231850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:07.232457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:07.232739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:07.233014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:07.234776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:07.246836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:07.248499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:07.258563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1148/2322] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:08.396262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3416b63-5f5e-440c-b964-a880d67a1a0c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:08.396726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3416b63-5f5e-440c-b964-a880d67a1a0c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:08.396832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3416b63-5f5e-440c-b964-a880d67a1a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:08.416083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b3416b63-5f5e-440c-b964-a880d67a1a0c None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:08.417309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b3416b63-5f5e-440c-b964-a880d67a1a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:08.425076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1175/2323] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:51:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:51:08.438134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:08.448562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520016a-6e45-4c33-babb-386194a81742 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:08.453478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:08.454094 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:08.454378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:08.454656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:08.463466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:08.465042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:08.473809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1149/2324] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:51:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:51:10.860782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:10.861305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:10.862173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:10.885974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] MFA Rules not processed for user `1c081d9bf3744177a2d2d50962a38027`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:10.887342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:10.894879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1176/2325] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:51:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:51:11.553181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:11.563655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefc1be-a031-453c-a882-d52b2a972a34 service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:11.568124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:11.568765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:11.569065 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:11.569400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:11.571316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:11.583694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:11.585464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:11.595000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1150/2326] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:12.595735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:12.631673 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0551a74-305a-47da-a68b-c651d58953e1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:12.652037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:12.652751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:12.653030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:12.653296 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:12.655308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:12.671730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:12.673532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:12.685734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1177/2327] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:14.537687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:14.551533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d510f-9ff0-4555-84c9-932c649ecd39 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:14.556952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:14.557537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:14.557824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:14.558145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:14.559853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:14.616729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:14.618349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:14.627388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1151/2328] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:22.681174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=24 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:23.571431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:23.582412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d31b0-0418-4a88-af3b-628960e2f104 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:23.587793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:23.588387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:23.588689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:23.588957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:23.590683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:23.602798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:23.604382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:23.617371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1178/2329] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:24.623157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:33.610461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:39.897411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:39.910041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a21311-16cd-400c-8066-b65eab5c52cd service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:39.915722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:39.916333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:39.916614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:39.916900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:39.918822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:39.932280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:39.933807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:39.944715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1152/2330] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:44.417624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:44.427650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ddff2-4639-412c-998a-649c4ceedbf1 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:44.431553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:44.432250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:44.432650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:44.432993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:44.442337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:44.443744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2f4a-cb3a-41ab-9c7d-2d5fc4ae3500 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:44.453682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1179/2331] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:49.163126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:49.175209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf99cd-6735-4c3d-b6c5-2079f5c6f459 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:49.180694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:49.181290 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:49.181571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:49.181842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:49.183585 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:49.244780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:49.246396 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c23193-2792-4efe-aaa6-431421cc526f service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:49.255422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1153/2332] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:51:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:51:50.425448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:50.425977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:50.425977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:50.444941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] MFA Rules not processed for user `41c9e5d537ab4a4a8931311d2dff3f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:50.446173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:50.452877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1180/2333] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:51:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:51:59.251329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=34 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:00.440151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=9 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:37.897836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164555fa-bb2e-4901-8fbc-9f67af98f091 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.898401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164555fa-bb2e-4901-8fbc-9f67af98f091 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.898499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164555fa-bb2e-4901-8fbc-9f67af98f091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.917789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-164555fa-bb2e-4901-8fbc-9f67af98f091 None None] MFA Rules not processed for user `e00238b83af948fe8752f998aa08b0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:37.919036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-164555fa-bb2e-4901-8fbc-9f67af98f091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:37.925854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1154/2334] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:52:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:52:47.912300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.783339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.796885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38302e9f-1d37-4352-9533-9a3a69c8e4aa None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.802320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.802922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.803197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.803468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.805857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:05.816515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.818208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23122a-4401-4e4e-b2d9-133bfa755261 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.827232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1181/2335] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:15.823068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.137931 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf0a6-6d11-4bd7-ac46-36ef1d27d119 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.138454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf0a6-6d11-4bd7-ac46-36ef1d27d119 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.138581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf0a6-6d11-4bd7-ac46-36ef1d27d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.160062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-25dbf0a6-6d11-4bd7-ac46-36ef1d27d119 None None] MFA Rules not processed for user `b193d0924fb448bfac2027034cb8dfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:24.160062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-25dbf0a6-6d11-4bd7-ac46-36ef1d27d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:24.165871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1155/2336] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:34.151778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.367557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.368094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.368094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.403859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] MFA Rules not processed for user `683fe1045ddb4bc1a8a94d3948495f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:37.404809 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:37.437460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1182/2337] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:43.314902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.315485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.317647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.348964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:43.350387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:43.392000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1156/2338] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:43.523015 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.552358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f208f25-159c-4416-ace5-9de22c472335 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.557900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.558570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.558870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.559180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.568749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.570458 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.579733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1183/2339] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 11:53:44.785003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.799536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5d3a-46f9-4fd6-9828-a0e0aa892ff8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.801722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.802533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.802914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.803303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.806170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:44.895382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.896849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.916894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1157/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:45.140898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.160571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f37cd-d72c-4928-be8d-7856abe4d986 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.164671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.165408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.165793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.167080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.168928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:45.185006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.186966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.200709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1184/2341] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:49.013090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.086664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c607677-7fdf-40b9-97e5-801430802e25 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.101824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.102540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.102821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.103106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.105360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:49.169986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.172228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.183470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1158/2342] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:50.910528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.910528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:50.916748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e723b8-2ac2-4345-8a7d-edc65a6ae43d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.922142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.922845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.923191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.923461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] PATH_INFO: `/v3/users/41c9e5d537ab4a4a8931311d2dff3f13` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.924593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c9e5d537ab4a4a8931311d2dff3f13)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.926619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.165466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c9e5d537ab4a4a8931311d2dff3f13'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:51.179703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c9e5d537ab4a4a8931311d2dff3f13'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:51.187026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c9e5d537ab4a4a8931311d2dff3f13'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:51.187768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c9e5d537ab4a4a8931311d2dff3f13'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:51.273498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-08073261-2d0c-43f2-a17d-b45475e203a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c9e5d537ab4a4a8931311d2dff3f13'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:51.282425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1185/2343] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:53:50 2022] DELETE /identity/v3/users/41c9e5d537ab4a4a8931311d2dff3f13 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:51.754593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.790440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275e3f-941a-4853-aef5-7ee40e1d0506 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.863561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.871227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30797812-86a8-4990-8861-f62685284979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.871946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30797812-86a8-4990-8861-f62685284979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.872353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30797812-86a8-4990-8861-f62685284979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.872655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30797812-86a8-4990-8861-f62685284979 admin admin] PATH_INFO: `/v3/projects/05fdf6e9eb54460aa442c83706b1238e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.873983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30797812-86a8-4990-8861-f62685284979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05fdf6e9eb54460aa442c83706b1238e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.876577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30797812-86a8-4990-8861-f62685284979 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.014119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05fdf6e9eb54460aa442c83706b1238e', 'request_id': 'req-30797812-86a8-4990-8861-f62685284979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:52.125782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05fdf6e9eb54460aa442c83706b1238e', 'request_id': 'req-30797812-86a8-4990-8861-f62685284979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:52.134877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05fdf6e9eb54460aa442c83706b1238e', 'request_id': 'req-30797812-86a8-4990-8861-f62685284979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:52.148024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] The token cache is being invalidate because project 05fdf6e9eb54460aa442c83706b1238e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:53:52.148306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:52.149158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-30797812-86a8-4990-8861-f62685284979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:52.151143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1159/2344] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:53:51 2022] DELETE /identity/v3/projects/05fdf6e9eb54460aa442c83706b1238e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:52.161031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.161850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.162161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.230847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:52.232091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:52.270942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.301401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1186/2345] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:52.309964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.310283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.310555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.331500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:52.332705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:52.340819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1160/2346] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:52.345141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.354748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c58247-39ae-4f0e-9f25-cd2a4215bfa9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.359773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.360371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.360647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.360917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.361560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:52.361863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.363562 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.375503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1187/2347] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:52.379805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.388856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ebe9b-17e4-41ae-8481-75a24c8ccedd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.393602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.394250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.394527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.394793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.395423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.397354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.508965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1161/2348] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:52.513433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.524323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b5633-8da7-49fd-ac0a-277cbbd06a1c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.593068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.594512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.594787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.595201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.595869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.597970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.715066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1188/2349] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/users => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:53:52.719575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.730626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2c42c-f80b-467a-9fe8-4e333f1b3059 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.736745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.737231 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.737513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.737790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.738514 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.740194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.751867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1162/2350] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:52.755949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.766908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0c88-7f54-430c-9aa1-83ca78422090 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.772795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.773435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.773716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.773992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.774704 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.776335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b63d9e-e9d9-446d-ac70-1abaab22013f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.788849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1189/2351] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:52.793263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.805416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12075552-c69e-4e31-9899-734e7797ee23 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.810353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.811099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.811450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.811789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] PATH_INFO: `/v3/projects/6adfc2b32ebf40c380e619761723970f/users/b77eee6d144f42749b7ae00878fbb254/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.830394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adfc2b32ebf40c380e619761723970f, user_id=b77eee6d144f42749b7ae00878fbb254, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.833196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.901554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1163/2352] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:52 2022] PUT /identity/v3/projects/6adfc2b32ebf40c380e619761723970f/users/b77eee6d144f42749b7ae00878fbb254/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:52.912908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.912908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.912908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.948602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] MFA Rules not processed for user `b77eee6d144f42749b7ae00878fbb254`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:52.948602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:53.061832 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1190/2353] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:53.064380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.167853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b258a5-da51-466a-842e-f9bb85912629 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.174872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.175496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.176160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.176439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.177704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.179778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.323141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1164/2354] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:53:53 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:53.333972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.347023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec87cb-4886-46e1-b0bf-ea3c3615de38 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.383421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.413343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.413969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.414280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.414525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.415250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.417166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.504648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1191/2355] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:53 2022] POST /identity/v3/users => generated 309 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:53.506258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.517634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49351-ca52-4c72-9d74-a0f0a3a57234 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.523030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.528613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.540788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1165/2356] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:53.544941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.557308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada6408-27d8-40c0-9b77-8facce1932ab admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.562908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.564242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.564528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.564845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] PATH_INFO: `/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a/users/967a00a51c224e5a8090312854dae9ac/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.585236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72107cf8e97842a08cdbe4548f2f2c7a, user_id=967a00a51c224e5a8090312854dae9ac, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.586025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.686306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1192/2357] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:53 2022] PUT /identity/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a/users/967a00a51c224e5a8090312854dae9ac/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:53.690092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.702009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db25cd8-e07b-4c15-b61c-aad769aedd74 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.766619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.767238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.767517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.767790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.768468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.770085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc82126-6280-4535-9e79-7a12047d3ecb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.784129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1166/2358] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:53.785308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.795842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3727fd-db7c-43ec-afc0-196dfb5f9b8e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.800639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.801300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.803097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.803097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] PATH_INFO: `/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a/users/967a00a51c224e5a8090312854dae9ac/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.804071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72107cf8e97842a08cdbe4548f2f2c7a, user_id=967a00a51c224e5a8090312854dae9ac, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.806846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.818344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1193/2359] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:53 2022] PUT /identity/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a/users/967a00a51c224e5a8090312854dae9ac/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:53.829062 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.830200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.830368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.859193 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] MFA Rules not processed for user `967a00a51c224e5a8090312854dae9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:53.860417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:53.935206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1167/2360] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:53.942914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.945234 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.023070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227bdd3-bdf4-4ed5-88c7-3e22602d0ef6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.026878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.027481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.027752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.028022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.038100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.042154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.060112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1194/2361] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:54.119566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.121116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.204499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a969b-c48f-4b2b-b3c3-f1eec6272b73 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.210267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.211234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.211234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.211448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.213264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.219730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.226731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.228951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.240099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1168/2362] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:54.262542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.262542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.280632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.293085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c15-32ca-4139-91af-ea0ab5d87a37 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.298388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.299087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.299370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.299642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] PATH_INFO: `/v3/users/b77eee6d144f42749b7ae00878fbb254` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.300773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77eee6d144f42749b7ae00878fbb254)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.302785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.338412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3b1a1-155b-46d0-a66f-bae77e8554b9 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.344277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.344909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.345191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.345466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] PATH_INFO: `/v3/users/b9a4f3995e1242de9d4bc87ce0608265` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.346739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a4f3995e1242de9d4bc87ce0608265)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.348981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.401729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.509541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77eee6d144f42749b7ae00878fbb254'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.522405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77eee6d144f42749b7ae00878fbb254'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.529306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77eee6d144f42749b7ae00878fbb254'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.529986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77eee6d144f42749b7ae00878fbb254'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.548807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a4f3995e1242de9d4bc87ce0608265'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.561123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a4f3995e1242de9d4bc87ce0608265'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.568223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a4f3995e1242de9d4bc87ce0608265'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.619951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a6c1f2db-afa6-4dca-9f37-e43f557451cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77eee6d144f42749b7ae00878fbb254'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.626090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1195/2363] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:53:54 2022] DELETE /identity/v3/users/b77eee6d144f42749b7ae00878fbb254 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:54.630405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.630688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.630959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.650327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:54.651526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.671975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a4f3995e1242de9d4bc87ce0608265'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.672611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a4f3995e1242de9d4bc87ce0608265'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:54.679491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1169/2364] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:53:54 2022] DELETE /identity/v3/users/b9a4f3995e1242de9d4bc87ce0608265 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:54.713447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1196/2365] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:53:54.723951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.725473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.808465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115979-ae0a-4934-8c39-c62f5f7bf2fc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.815521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.816601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.817046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.817384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.842865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.889497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.891622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.901400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1170/2366] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:54.965874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.967309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.981319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748095be-3642-440b-bfda-7bd7600615f0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.000427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.001549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.001934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.002096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] PATH_INFO: `/v3/projects/4395b654958545cc8913f4d7786c736a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.003268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4395b654958545cc8913f4d7786c736a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.005764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.184805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4395b654958545cc8913f4d7786c736a', 'request_id': 'req-3112b031-a32a-43b0-bebd-b9239746a9b7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.193611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4395b654958545cc8913f4d7786c736a', 'request_id': 'req-3112b031-a32a-43b0-bebd-b9239746a9b7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.200285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4395b654958545cc8913f4d7786c736a', 'request_id': 'req-3112b031-a32a-43b0-bebd-b9239746a9b7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.353549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] The token cache is being invalidate because project 4395b654958545cc8913f4d7786c736a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:53:55.353862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.354505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.355995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1197/2367] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:53:54 2022] DELETE /identity/v3/projects/4395b654958545cc8913f4d7786c736a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:55.360499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.370903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055d5d-302b-457d-8bc5-891b85f21326 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.427916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.428470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.428749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.429022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] PATH_INFO: `/v3/users/e4acdc49a7e0499db338bb50eec8d7ae` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.430195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4acdc49a7e0499db338bb50eec8d7ae)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.432270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.443741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.444990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:55.458719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112b031-a32a-43b0-bebd-b9239746a9b7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.465329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.466465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.467027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.467485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] PATH_INFO: `/v3/projects/6adfc2b32ebf40c380e619761723970f` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.468980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6adfc2b32ebf40c380e619761723970f)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.471140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.501035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:55.644326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6adfc2b32ebf40c380e619761723970f', 'request_id': 'req-66a3df57-37dc-4d25-b420-1b95281cb182'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.653519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6adfc2b32ebf40c380e619761723970f', 'request_id': 'req-66a3df57-37dc-4d25-b420-1b95281cb182'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.661496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6adfc2b32ebf40c380e619761723970f', 'request_id': 'req-66a3df57-37dc-4d25-b420-1b95281cb182'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.668097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4acdc49a7e0499db338bb50eec8d7ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.683815 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4acdc49a7e0499db338bb50eec8d7ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.691812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4acdc49a7e0499db338bb50eec8d7ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.764654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] The token cache is being invalidate because project 6adfc2b32ebf40c380e619761723970f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:53:55.765378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.765794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.767571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1198/2368] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:53:55 2022] DELETE /identity/v3/projects/6adfc2b32ebf40c380e619761723970f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:55.771349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.772710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:55.828189 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4acdc49a7e0499db338bb50eec8d7ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.828837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4acdc49a7e0499db338bb50eec8d7ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:55.839111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1171/2369] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:53:55 2022] DELETE /identity/v3/users/e4acdc49a7e0499db338bb50eec8d7ae => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:55.841216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3df57-37dc-4d25-b420-1b95281cb182 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.912326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.912976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.913259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.913555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] PATH_INFO: `/v3/users/967a00a51c224e5a8090312854dae9ac` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.914726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] RBAC: Authorizing `identity:delete_user(user_id=967a00a51c224e5a8090312854dae9ac)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.916752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.151761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967a00a51c224e5a8090312854dae9ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.165855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.170930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967a00a51c224e5a8090312854dae9ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.177273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.178771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:56.195254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967a00a51c224e5a8090312854dae9ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.198540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967a00a51c224e5a8090312854dae9ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.206847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510bd7f-d5b0-47c7-9324-3c35a1cd1633 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.217491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.217491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.217491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.217491 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] PATH_INFO: `/v3/projects/763412ca2259491ebfac445f6369c3f4` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.218134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=763412ca2259491ebfac445f6369c3f4)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.218134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.296701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967a00a51c224e5a8090312854dae9ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.304187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1199/2370] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:53:55 2022] DELETE /identity/v3/users/967a00a51c224e5a8090312854dae9ac => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:56.332494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.347710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763412ca2259491ebfac445f6369c3f4', 'request_id': 'req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.415392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763412ca2259491ebfac445f6369c3f4', 'request_id': 'req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.421472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763412ca2259491ebfac445f6369c3f4', 'request_id': 'req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.432997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] The token cache is being invalidate because project 763412ca2259491ebfac445f6369c3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:53:56.433285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.433929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0d4274c2-e61f-4a1b-9802-1cf20b4c7979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:56.435683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1172/2371] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:53:56 2022] DELETE /identity/v3/projects/763412ca2259491ebfac445f6369c3f4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:56.819663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.821341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:56.892104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb95af-ef27-4aca-ab8e-e006f450618d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.948895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.949533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.949813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.950123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] PATH_INFO: `/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.951290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] RBAC: Authorizing `identity:delete_project(project_id=72107cf8e97842a08cdbe4548f2f2c7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.953417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.082232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72107cf8e97842a08cdbe4548f2f2c7a', 'request_id': 'req-57f4f64b-fb53-4bcc-942d-a960b8eda5af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:57.089117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72107cf8e97842a08cdbe4548f2f2c7a', 'request_id': 'req-57f4f64b-fb53-4bcc-942d-a960b8eda5af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:57.095568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72107cf8e97842a08cdbe4548f2f2c7a', 'request_id': 'req-57f4f64b-fb53-4bcc-942d-a960b8eda5af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:57.171475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] The token cache is being invalidate because project 72107cf8e97842a08cdbe4548f2f2c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:53:57.173604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:57.173604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57f4f64b-fb53-4bcc-942d-a960b8eda5af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:53:57.175089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1200/2372] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:53:56 2022] DELETE /identity/v3/projects/72107cf8e97842a08cdbe4548f2f2c7a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:57.180339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.180808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.181259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.207067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:57.208621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:57.283008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1173/2373] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:57.292038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.292389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.292742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.313863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:57.315128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:57.323741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1201/2374] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:57.328388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.337863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad5316-eade-4eb0-afe8-1327cc4410c0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.343533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.344300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.344654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.345229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.345947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:57.346355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.348056 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.362981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1174/2375] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:57.368273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.385447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60f038-4429-4666-a7aa-725a192aca22 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.390300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.391480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.391757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.392027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.392670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.394599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.400323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.553456 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1202/2376] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:57.554599 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.565551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce69e-0efd-4c6e-a750-49a643ee5194 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.629013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.629890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.630332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.630704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.631438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.633558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.744688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1175/2377] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:57.749005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.761316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a386-6cd8-4f5c-bb69-79aee4a415c7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.767060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.767865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.768309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.768766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.769573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.771534 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.785733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1203/2378] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:57.789873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.800978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222273-d408-4ef7-9b6a-6569d6bf3228 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.806592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.807280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.807660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.808001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.808722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.810450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a604c-5a26-4023-ae09-9f147c9285dc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.823765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1176/2379] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:53:57.828212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.838313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058f3b8-f6c6-480e-ae39-8466806c8faf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.842918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.843556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.844457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.844726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] PATH_INFO: `/v3/projects/669aae2aff9a490caa4b61aa4ae9a27f/users/0d3d2c09799846ac84f369b69cdcc4a8/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.860040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] RBAC: Authorizing `identity:create_grant(project_id=669aae2aff9a490caa4b61aa4ae9a27f, user_id=0d3d2c09799846ac84f369b69cdcc4a8, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.862806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.994621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1204/2380] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:57 2022] PUT /identity/v3/projects/669aae2aff9a490caa4b61aa4ae9a27f/users/0d3d2c09799846ac84f369b69cdcc4a8/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:58.004870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.005153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.005425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.035858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] MFA Rules not processed for user `0d3d2c09799846ac84f369b69cdcc4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:58.038206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:58.050598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:58.110826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1177/2381] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:53:58.115377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.179939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867437-530a-4c59-a205-03fc63fd801b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.184708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.185316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.185595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.185868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.186545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.188594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.321256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1205/2382] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:53:58 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:58.325329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.336759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17881069-b0d9-4b91-928b-6f5fec46743a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.400469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.401071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.401345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.401634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.402279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.404217 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.523503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1178/2383] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:58 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:53:58.527571 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.538434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb19089-b2e1-44b9-9270-896080d7d1d5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.542960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.543548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.543827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.544097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.544739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.546349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.556664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1206/2384] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:58.560659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.571373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2926b-48e6-4919-9eb7-56d29c532451 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.575872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.576504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.576785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.577081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] PATH_INFO: `/v3/projects/4d4ad65896ca4668926536840c07d6c9/users/957971184e004bcfbad687f35736e0ff/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.592077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d4ad65896ca4668926536840c07d6c9, user_id=957971184e004bcfbad687f35736e0ff, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.594860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.666075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1179/2385] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:58 2022] PUT /identity/v3/projects/4d4ad65896ca4668926536840c07d6c9/users/957971184e004bcfbad687f35736e0ff/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:58.670032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.679972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655cbd5-b23d-4945-9b93-cbd503aa4875 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.745175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.745779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.746085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.746351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.747004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.748576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae834a-45e4-460a-811e-db8fe1727d26 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.758969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1207/2386] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:53:58.763151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.774172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539877-6308-4251-99d8-0b44979ede8f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.778874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.779675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.779959 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.780236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] PATH_INFO: `/v3/projects/4d4ad65896ca4668926536840c07d6c9/users/957971184e004bcfbad687f35736e0ff/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.782272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d4ad65896ca4668926536840c07d6c9, user_id=957971184e004bcfbad687f35736e0ff, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.784925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.795174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1180/2387] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:53:58 2022] PUT /identity/v3/projects/4d4ad65896ca4668926536840c07d6c9/users/957971184e004bcfbad687f35736e0ff/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:58.805237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.805525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.805800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.834436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] MFA Rules not processed for user `957971184e004bcfbad687f35736e0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:58.835601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:58.844604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:58.901871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1208/2388] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:53:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:53:58.913216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:58.914972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:58.972168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acaa68e-baf6-499c-a052-54de89c8950e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:58.975850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:58.976445 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.976754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.977029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.986410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:58.987836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:58.997026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1181/2389] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:53:59.229989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:59.240086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac3103-695c-4046-968f-d773aca6b351 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:59.243680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:59.244274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.244554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.244848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.300413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:59.301825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2aac9-ae09-49a4-a976-0586da79db8f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:59.310560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1209/2390] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:53:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:08.992982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:09.306589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=52 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:12.364081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.425247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72421e2-958f-4653-8880-f98cb20e666a service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.430299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.430913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.431188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.431457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] PATH_INFO: `/v3/users/0d3d2c09799846ac84f369b69cdcc4a8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.432557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3d2c09799846ac84f369b69cdcc4a8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.434584 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.642965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3d2c09799846ac84f369b69cdcc4a8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:12.654814 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3d2c09799846ac84f369b69cdcc4a8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:12.661636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3d2c09799846ac84f369b69cdcc4a8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:12.736023 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3d2c09799846ac84f369b69cdcc4a8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:12.736637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3d2c09799846ac84f369b69cdcc4a8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:12.743259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1182/2391] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:12 2022] DELETE /identity/v3/users/0d3d2c09799846ac84f369b69cdcc4a8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:12.748018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.748513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.748614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.767410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:12.768624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:12.827780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1210/2392] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:12.837606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.839022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:12.895165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaae22-16c6-4987-8ed1-42be90351db9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.898728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.899323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.899604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.899878 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.908939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.910399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.921478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1183/2393] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:13.333937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:13.335929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:13.349662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e99d2c-3bf1-49ab-8c60-035fe243ce29 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:13.354977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:13.361153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:13.361153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:13.361153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] PATH_INFO: `/v3/projects/669aae2aff9a490caa4b61aa4ae9a27f` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:13.361153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669aae2aff9a490caa4b61aa4ae9a27f)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:13.361153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:13.480754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669aae2aff9a490caa4b61aa4ae9a27f', 'request_id': 'req-d4ee7050-3929-4495-ab53-f5c13dafe347'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.487193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669aae2aff9a490caa4b61aa4ae9a27f', 'request_id': 'req-d4ee7050-3929-4495-ab53-f5c13dafe347'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.492769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669aae2aff9a490caa4b61aa4ae9a27f', 'request_id': 'req-d4ee7050-3929-4495-ab53-f5c13dafe347'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.571988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] The token cache is being invalidate because project 669aae2aff9a490caa4b61aa4ae9a27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:13.572275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.572895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.574438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1211/2394] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:13 2022] DELETE /identity/v3/projects/669aae2aff9a490caa4b61aa4ae9a27f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:13.578551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:13.588852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281510e-9bdb-4325-9aca-a1b2abbbb672 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:13.606666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:13.643766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:13.644386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:13.644665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:13.644937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] PATH_INFO: `/v3/users/957971184e004bcfbad687f35736e0ff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:13.646044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957971184e004bcfbad687f35736e0ff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:13.648557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:13.843819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957971184e004bcfbad687f35736e0ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.855729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957971184e004bcfbad687f35736e0ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.864486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957971184e004bcfbad687f35736e0ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.942177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957971184e004bcfbad687f35736e0ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.942779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f0a50f7-d9a5-4435-9146-70a4d89d4fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957971184e004bcfbad687f35736e0ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:13.949806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1184/2395] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:13 2022] DELETE /identity/v3/users/957971184e004bcfbad687f35736e0ff => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:14.355623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.358275 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:14.371630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7050-3929-4495-ab53-f5c13dafe347 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.432227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.432869 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.433130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.433402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] PATH_INFO: `/v3/projects/4d4ad65896ca4668926536840c07d6c9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.434564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d4ad65896ca4668926536840c07d6c9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.436460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.522186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:14.538392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d4ad65896ca4668926536840c07d6c9', 'request_id': 'req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:14.544423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d4ad65896ca4668926536840c07d6c9', 'request_id': 'req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:14.549789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d4ad65896ca4668926536840c07d6c9', 'request_id': 'req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:14.638932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] The token cache is being invalidate because project 4d4ad65896ca4668926536840c07d6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:14.639220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:14.639850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7dc330e0-f8ff-4ae3-b500-23cbc442bdfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:14.641351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1212/2396] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:14 2022] DELETE /identity/v3/projects/4d4ad65896ca4668926536840c07d6c9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:14.646191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.647138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.647251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.668364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:14.672947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:14.727683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1185/2397] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:14.735818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.736094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.736390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.755733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:14.756939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:14.764574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1213/2398] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:14.768759 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.778915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e7b75-fd22-4583-af73-cc9d6868e5a1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.783743 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.784331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.784611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.784884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.785528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:14.785830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.788132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.799523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1186/2399] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:14.803905 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.812919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd66926-ce7d-4ec3-9928-b4084537bc7e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.817493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.818092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.818385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.818675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.819304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.821408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.939689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1214/2400] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:14 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:14.944004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.955227 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146ca4-0ebd-44e2-bd5e-02a8993f9207 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.982083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:15.012613 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.013223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.013501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.013770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.014441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.016351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.115504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1187/2401] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:14 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:15.119713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.129954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da24c8-c607-413f-bd2a-70b76b2938af admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.135067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.135714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.135983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.136291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.136923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.138654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.149134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1215/2402] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:15.153082 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.163152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0fc9c-229c-4bf2-9b80-801616173579 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.167763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.168348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.169832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.169832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.169832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.171273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3293a18-792e-4882-8cff-8cd8f1c024e6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.182327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1188/2403] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:15.186400 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.197583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22969f10-a70f-4c6d-9d1b-028b6682bc04 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.202168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.202807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.203085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.203361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] PATH_INFO: `/v3/projects/c245ae61e0364b0cb933d6c058079aec/users/1f38b079f7fb419993bedd02f3a1b80e/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.218484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c245ae61e0364b0cb933d6c058079aec, user_id=1f38b079f7fb419993bedd02f3a1b80e, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.221090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.286349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1216/2404] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:15 2022] PUT /identity/v3/projects/c245ae61e0364b0cb933d6c058079aec/users/1f38b079f7fb419993bedd02f3a1b80e/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:15.296636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.296983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.297275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.325711 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] MFA Rules not processed for user `1f38b079f7fb419993bedd02f3a1b80e`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:15.326964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:15.390933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1189/2405] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:15.407279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.472004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d4b56-b4b1-49b3-9b29-a386ab08f74e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.476682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.477310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.477586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.477857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.478548 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.480450 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.579885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1217/2406] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:15 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:15.583960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.594627 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4526c-8633-4f38-9d3c-716888bc3940 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.649765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.650400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.650660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.650930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.651560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.653483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.755434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1190/2407] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:15 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:15.759309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.769690 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f1d99-0cc8-43bf-884f-dd46759770a5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.774363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.774952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.775286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.775502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.776144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.777708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.788514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1218/2408] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:15.792483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.802758 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271418a-b485-4303-bdfd-7cf7029cf963 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.808069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.808657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.808937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.809210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.809854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.811500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e4c4d-22cb-4bfb-8c00-7e9af9787dc4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.821208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1191/2409] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:15.825419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.835457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19105960-1742-4d7e-a8be-73c00d55fc43 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.839996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.840636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.840914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.841188 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] PATH_INFO: `/v3/projects/48fb2d94614641f694d0986e5a088e91/users/fdc2ce2119be49938ccc594a7d2cc2f6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.855814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fb2d94614641f694d0986e5a088e91, user_id=fdc2ce2119be49938ccc594a7d2cc2f6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.858490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.060720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:16.065415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1219/2410] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:15 2022] PUT /identity/v3/projects/48fb2d94614641f694d0986e5a088e91/users/fdc2ce2119be49938ccc594a7d2cc2f6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:16.076943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.077225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.077496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.105747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] MFA Rules not processed for user `fdc2ce2119be49938ccc594a7d2cc2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:16.107000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:16.111148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:16.170647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1192/2411] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:16.175176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.235703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefe429-a114-41c1-87b6-49b15f0e5125 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.240893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.241494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.241771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.242043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.242698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.244581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.360864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1220/2412] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:16 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:16.365127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.376143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7eab-2776-414b-b8d2-770b6cac7243 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.432950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.433563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.433974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.434305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.434934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.436845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.554622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1193/2413] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:16 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:16.558566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.569439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441619a6-c82c-4f65-a416-ef3a24800f43 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.574396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.575034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.575310 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.575582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.576225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.577796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.588890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1221/2414] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:16.595026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.606390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20644f1-cacd-4ce8-bff9-0c3c201d3546 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.611079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.611663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.611939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.612207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] PATH_INFO: `/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d/users/828dab84cb09470aae2b63d74c661642/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.626752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b5e044539b481799eb0b4fdd4cb93d, user_id=828dab84cb09470aae2b63d74c661642, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.629480 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.702523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1194/2415] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:16 2022] PUT /identity/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d/users/828dab84cb09470aae2b63d74c661642/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:16.706489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.717166 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eb841-16a9-4563-9cf0-7853771cbeb8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.777018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.777648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.777928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.778231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.778872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.780547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b93ae-6a55-4cd8-8f5d-56924fb5be8a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.790582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1222/2416] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:16.794716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.804981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d66701-a570-4747-a7a7-002896f1baab admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:16.809813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:16.810489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.810765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.811081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] PATH_INFO: `/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d/users/828dab84cb09470aae2b63d74c661642/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.812967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b5e044539b481799eb0b4fdd4cb93d, user_id=828dab84cb09470aae2b63d74c661642, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:16.815876 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:16.827454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1195/2417] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:16 2022] PUT /identity/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d/users/828dab84cb09470aae2b63d74c661642/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:16.840032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.840808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.840808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.869381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] MFA Rules not processed for user `828dab84cb09470aae2b63d74c661642`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:16.870569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:16.928840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:16.937220 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1223/2418] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:16.949888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:16.951133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:17.019234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:17.033922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4887c-7ef4-434f-9643-e89f275d916d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:17.039596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:17.040264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.040539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.040802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.050376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:17.051807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:17.061732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1196/2419] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:18.159734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:18.161148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:18.172984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a31d99-ce3d-47ec-a440-6f4cf2a13743 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:18.176496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:18.177085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.177361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.177631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.233951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:18.235413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:18.244280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1224/2420] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:25.206637 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:25.216101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342bcd6-e278-453c-850f-5dd582d268e7 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.219803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.220368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.220644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.220913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.277472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.278953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332870-17c0-400d-80fb-88a08a8a069b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.287820 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1197/2421] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:28.240129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=30 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:29.538407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:29.601160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cf9a6-5d41-4372-9233-9d057934e7ca service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:29.606701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:29.607514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:29.607800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:29.608074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] PATH_INFO: `/v3/users/1f38b079f7fb419993bedd02f3a1b80e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:29.609235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f38b079f7fb419993bedd02f3a1b80e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:29.611214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:29.925944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f38b079f7fb419993bedd02f3a1b80e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:29.941074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f38b079f7fb419993bedd02f3a1b80e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:29.949544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f38b079f7fb419993bedd02f3a1b80e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:29.950532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f38b079f7fb419993bedd02f3a1b80e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:30.042965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f38b079f7fb419993bedd02f3a1b80e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:30.049313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1225/2422] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:29 2022] DELETE /identity/v3/users/1f38b079f7fb419993bedd02f3a1b80e => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:30.056300 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.056777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.056970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.076670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:30.077892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:30.148569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1198/2423] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:30.158199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:30.159734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:30.217892 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c269ce-01fc-4cf9-8b5e-79c377cf6950 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:30.221986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:30.226700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.226700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.226700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.233405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:30.234984 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:30.245294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1226/2424] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:30.786987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:30.790802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:30.809801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed4a3-440b-4ba3-822f-4464bd5babc3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:30.816590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:30.816590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.816590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.817434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] PATH_INFO: `/v3/projects/c245ae61e0364b0cb933d6c058079aec` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.820386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c245ae61e0364b0cb933d6c058079aec)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:30.823343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:30.970558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c245ae61e0364b0cb933d6c058079aec', 'request_id': 'req-38f877ca-1358-485c-a001-c69076f3e5ac'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.052448 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c245ae61e0364b0cb933d6c058079aec', 'request_id': 'req-38f877ca-1358-485c-a001-c69076f3e5ac'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.059871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c245ae61e0364b0cb933d6c058079aec', 'request_id': 'req-38f877ca-1358-485c-a001-c69076f3e5ac'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.072804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] The token cache is being invalidate because project c245ae61e0364b0cb933d6c058079aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:31.073349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.074292 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.076019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1199/2425] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:30 2022] DELETE /identity/v3/projects/c245ae61e0364b0cb933d6c058079aec => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:31.080494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.092393 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e622-36e7-48da-baa6-2b513a920b79 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.119661 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.177707 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.181878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.182389 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.182921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] PATH_INFO: `/v3/users/fdc2ce2119be49938ccc594a7d2cc2f6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.184352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc2ce2119be49938ccc594a7d2cc2f6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.186556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.407332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc2ce2119be49938ccc594a7d2cc2f6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.420576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc2ce2119be49938ccc594a7d2cc2f6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.427538 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc2ce2119be49938ccc594a7d2cc2f6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.428245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc2ce2119be49938ccc594a7d2cc2f6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.455673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.455984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:31.461441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.511687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc2ce2119be49938ccc594a7d2cc2f6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:31.518710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1227/2426] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:31 2022] DELETE /identity/v3/users/fdc2ce2119be49938ccc594a7d2cc2f6 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:54:31.531522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f877ca-1358-485c-a001-c69076f3e5ac admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.603854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.604836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.609742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.609742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.609742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:31.623130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.624740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.635076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1200/2427] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:54:31.981766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.983200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:32.051625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fdf46-9eb8-40a7-ab2c-0b9176fe74c9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.056270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.056891 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.057173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.057448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] PATH_INFO: `/v3/projects/48fb2d94614641f694d0986e5a088e91` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.058586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48fb2d94614641f694d0986e5a088e91)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.060466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.155909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.162584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48fb2d94614641f694d0986e5a088e91', 'request_id': 'req-ebc4a60a-d528-494a-9882-f297b42d0f47'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.170001 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48fb2d94614641f694d0986e5a088e91', 'request_id': 'req-ebc4a60a-d528-494a-9882-f297b42d0f47'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.175658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48fb2d94614641f694d0986e5a088e91', 'request_id': 'req-ebc4a60a-d528-494a-9882-f297b42d0f47'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.265438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] The token cache is being invalidate because project 48fb2d94614641f694d0986e5a088e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:32.265728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.272561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.272561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1228/2428] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:31 2022] DELETE /identity/v3/projects/48fb2d94614641f694d0986e5a088e91 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:32.275928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.286535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b38376-ce49-47a4-aed4-63e0036964fd service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.342346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.342981 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.343264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.343541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] PATH_INFO: `/v3/users/828dab84cb09470aae2b63d74c661642` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.344685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828dab84cb09470aae2b63d74c661642)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.346636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.499813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.508819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828dab84cb09470aae2b63d74c661642'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.521051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828dab84cb09470aae2b63d74c661642'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.527978 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828dab84cb09470aae2b63d74c661642'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.613391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828dab84cb09470aae2b63d74c661642'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.614136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c98eb842-764c-43a8-812d-d971bd926080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828dab84cb09470aae2b63d74c661642'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:32.620320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1201/2429] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:32 2022] DELETE /identity/v3/users/828dab84cb09470aae2b63d74c661642 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:33.015906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.017402 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:33.031194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a60a-d528-494a-9882-f297b42d0f47 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.089305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.089964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.090549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.090824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] PATH_INFO: `/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.091956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b5e044539b481799eb0b4fdd4cb93d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.093852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.225935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b5e044539b481799eb0b4fdd4cb93d', 'request_id': 'req-0508804e-6e83-488d-ad68-e501877ba10f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:33.232239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b5e044539b481799eb0b4fdd4cb93d', 'request_id': 'req-0508804e-6e83-488d-ad68-e501877ba10f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:33.237889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b5e044539b481799eb0b4fdd4cb93d', 'request_id': 'req-0508804e-6e83-488d-ad68-e501877ba10f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:33.317467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29bb1-5c20-44d1-9952-83f349644a5a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.317467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29bb1-5c20-44d1-9952-83f349644a5a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.318422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29bb1-5c20-44d1-9952-83f349644a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.318706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] The token cache is being invalidate because project a4b5e044539b481799eb0b4fdd4cb93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:33.319104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:33.319861 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0508804e-6e83-488d-ad68-e501877ba10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:33.328823 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1229/2430] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:33 2022] DELETE /identity/v3/projects/a4b5e044539b481799eb0b4fdd4cb93d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:33.334651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.334935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.335207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.338095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-27c29bb1-5c20-44d1-9952-83f349644a5a None None] MFA Rules not processed for user `1c081d9bf3744177a2d2d50962a38027`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:33.339357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-27c29bb1-5c20-44d1-9952-83f349644a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:33.356214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:33.357426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:33.397790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1202/2431] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:33.421883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1230/2432] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:54:33.446426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.446426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.446426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.458022 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:33.459323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:33.467280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1203/2433] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:33.471818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.477156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:33.482614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf22e1-1904-4076-8d7a-179dafbaa68f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.487558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.488242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.493189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.493189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.493189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:33.493189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.493189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.504543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1231/2434] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:33.509582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.527664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728725d-6b75-4f88-bedf-ba808bc8bf16 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.541210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.541827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.542126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.542400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.543035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.544970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.635283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1204/2435] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:33 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:33.639162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.649814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89a360-1abe-4b2b-9f3d-922bb4854c85 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.718556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.719168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.719448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.719804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.720465 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.722390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.809991 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1232/2436] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:33 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:33.813908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.825076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe40b6e-0dee-4654-9809-dc3ae194db39 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.829804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.830443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.830721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.835498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.835498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.835498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.846588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1205/2437] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:33.847817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.858350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6808bf-c2dc-47d2-b202-b4454835af32 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.863182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.863786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.864068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.864342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.865024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.866712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923c34e-52bd-49df-9fb8-b14f710b25b9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:33.877950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1233/2438] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:33.881988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:33.892164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51129e-7b77-48bc-98c7-b6d302bee6a3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:33.897362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:33.898016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:33.898330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:33.898602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] PATH_INFO: `/v3/projects/248b754c2ad64bd8a4ee7b9ee8f40c80/users/2d50f60ade964694b9080327c9585524/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:33.913592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=248b754c2ad64bd8a4ee7b9ee8f40c80, user_id=2d50f60ade964694b9080327c9585524, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:33.916829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.013883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1206/2439] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:33 2022] PUT /identity/v3/projects/248b754c2ad64bd8a4ee7b9ee8f40c80/users/2d50f60ade964694b9080327c9585524/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:34.026671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.026962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.027239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.058656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] MFA Rules not processed for user `2d50f60ade964694b9080327c9585524`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:34.059858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:34.126592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1234/2440] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:34.131180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.188696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:34.193696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36712434-6ce7-438d-ac38-ac6a421901a8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.198713 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.199312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.199592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.199863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.200487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.202428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.367356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1207/2441] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:34 2022] POST /identity/v3/projects => generated 391 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:34.368752 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.380303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e313f4e-3028-4227-b2cd-ca9e5329fe5b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.432366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:34.438204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.442889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.442889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.442889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.442889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.442889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.616493 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1235/2442] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:34 2022] POST /identity/v3/users => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:34.620517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.631169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071ee3-d498-43b1-97fa-f35957aacf80 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.636837 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.639042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.639042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.639042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.639042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.640422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.651260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1208/2443] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:34.655261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.665572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a243-a348-4bcd-9ba9-e7ed51ef5a41 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.671052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.671989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.672280 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.672557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] PATH_INFO: `/v3/projects/b01761e14b734976816f0b3e017e4800/users/a6c1dd0d9f42458f984aa36b913e8110/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.687504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01761e14b734976816f0b3e017e4800, user_id=a6c1dd0d9f42458f984aa36b913e8110, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.690495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.764216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1236/2444] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:34 2022] PUT /identity/v3/projects/b01761e14b734976816f0b3e017e4800/users/a6c1dd0d9f42458f984aa36b913e8110/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:34.768125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.779138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854ae2-5067-434e-9084-c7036d40fc03 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.842626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.843239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.843520 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.843791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.844434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.846037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee56a-f026-45cd-994f-f983853aa30e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.860688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1209/2445] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:34.861919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:34.872627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bc41-2dac-436c-82a8-160f977e4aa4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:34.877730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:34.878441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.878713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.878978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] PATH_INFO: `/v3/projects/b01761e14b734976816f0b3e017e4800/users/a6c1dd0d9f42458f984aa36b913e8110/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.881033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01761e14b734976816f0b3e017e4800, user_id=a6c1dd0d9f42458f984aa36b913e8110, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:34.885116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:34.895161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1237/2446] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:34 2022] PUT /identity/v3/projects/b01761e14b734976816f0b3e017e4800/users/a6c1dd0d9f42458f984aa36b913e8110/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:34.905675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.905960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.906285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.936207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] MFA Rules not processed for user `a6c1dd0d9f42458f984aa36b913e8110`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:34.937438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:35.012668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1210/2447] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:35.023677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:35.025445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:35.085204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c305-2414-4ea7-8dcb-52fd8aee3fff admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:35.089180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:35.091434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:35.091434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:35.091434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:35.099482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:35.100949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:35.110624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1238/2448] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:35 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:35.167768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:35.167768 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:35.172554 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9594d92-3ef6-45a5-9fd1-956592553d85 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:35.176156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:35.176747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:35.177030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:35.177305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:35.226349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:35.236506 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:35.237983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47ca0a-0500-49f7-a844-f1d60dfcbe0c service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:35.247779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1211/2449] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:41.711738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:41.775780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c9a27-443d-4c70-858d-41d8d8907a69 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:41.780316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:41.780940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:41.781219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:41.781491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] PATH_INFO: `/v3/users/2d50f60ade964694b9080327c9585524` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:41.782650 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d50f60ade964694b9080327c9585524)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:41.784639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:41.982785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d50f60ade964694b9080327c9585524'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:41.988488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:41.996618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d50f60ade964694b9080327c9585524'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:42.004131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d50f60ade964694b9080327c9585524'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:42.005498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d50f60ade964694b9080327c9585524'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:42.087626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d50f60ade964694b9080327c9585524'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:42.093877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1239/2450] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:41 2022] DELETE /identity/v3/users/2d50f60ade964694b9080327c9585524 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:42.098590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:42.099072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:42.099173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:42.118659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:42.120507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:42.177654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1212/2451] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:42.188021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:42.189453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:42.252147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be4632-46bf-48ef-a17d-9f7903c66a30 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:42.255833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:42.256434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:42.256713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:42.256985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:42.265843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:42.267302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:42.276859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1240/2452] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:42.742628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:42.744181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:42.756819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c86b9-a20b-4d3c-a0b0-9398e36d7783 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:42.762471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:42.762936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:42.763222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:42.763518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] PATH_INFO: `/v3/projects/248b754c2ad64bd8a4ee7b9ee8f40c80` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:42.764662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=248b754c2ad64bd8a4ee7b9ee8f40c80)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:42.766647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:42.948688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248b754c2ad64bd8a4ee7b9ee8f40c80', 'request_id': 'req-ba014da5-ca6d-4a0d-8606-4e811470dc5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.027478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248b754c2ad64bd8a4ee7b9ee8f40c80', 'request_id': 'req-ba014da5-ca6d-4a0d-8606-4e811470dc5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.033694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248b754c2ad64bd8a4ee7b9ee8f40c80', 'request_id': 'req-ba014da5-ca6d-4a0d-8606-4e811470dc5b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.045771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] The token cache is being invalidate because project 248b754c2ad64bd8a4ee7b9ee8f40c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:43.046137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.046841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.048474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1213/2453] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:42 2022] DELETE /identity/v3/projects/248b754c2ad64bd8a4ee7b9ee8f40c80 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:43.052798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:43.064627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d6b77-a1ab-49ea-abf8-d2425adbbd77 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:43.122086 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:43.122893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:43.123272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:43.123646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] PATH_INFO: `/v3/users/a6c1dd0d9f42458f984aa36b913e8110` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:43.124906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c1dd0d9f42458f984aa36b913e8110)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:43.126959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:43.280455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c1dd0d9f42458f984aa36b913e8110'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.292359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c1dd0d9f42458f984aa36b913e8110'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.299223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c1dd0d9f42458f984aa36b913e8110'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.299894 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c1dd0d9f42458f984aa36b913e8110'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.302426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:43.375134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f9e1271-3313-4848-ba46-981a8f80ac7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c1dd0d9f42458f984aa36b913e8110'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:43.381567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1241/2454] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:54:43 2022] DELETE /identity/v3/users/a6c1dd0d9f42458f984aa36b913e8110 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:43.781451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:43.782992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:43.797854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba014da5-ca6d-4a0d-8606-4e811470dc5b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:43.856577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:43.857204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:43.857485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:43.857761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] PATH_INFO: `/v3/projects/b01761e14b734976816f0b3e017e4800` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:43.858915 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01761e14b734976816f0b3e017e4800)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:43.860828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:43.876671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:44.005730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01761e14b734976816f0b3e017e4800', 'request_id': 'req-796cab48-3461-43e9-9a3c-5ff8fbebdf95'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:44.093483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01761e14b734976816f0b3e017e4800', 'request_id': 'req-796cab48-3461-43e9-9a3c-5ff8fbebdf95'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:44.100986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01761e14b734976816f0b3e017e4800', 'request_id': 'req-796cab48-3461-43e9-9a3c-5ff8fbebdf95'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:44.114463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] The token cache is being invalidate because project b01761e14b734976816f0b3e017e4800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:54:44.114754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:44.115436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-796cab48-3461-43e9-9a3c-5ff8fbebdf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:54:44.116939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1214/2455] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:54:43 2022] DELETE /identity/v3/projects/b01761e14b734976816f0b3e017e4800 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:44.122513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.123157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.123448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.144122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:44.145334 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:44.205097 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1242/2456] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:54:44.271315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.271603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.271875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.293922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:44.295179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:44.303736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1215/2457] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:44.308446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.319010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29c9f8-f452-4656-b579-ec5d823da561 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.324936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.325492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.325772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.326047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.327235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:44.327235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.328675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.341961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1243/2458] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:44.346374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.357380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbf049-394a-4355-8d9c-9478623232b0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.363360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.363974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.364256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.364529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.365165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.367502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.497803 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1216/2459] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:44.503268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.513331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3cc5-6107-457d-9f49-f433a5939992 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.572148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.572756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.573066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.573341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.574122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.576335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.656106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1244/2460] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:44.660428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.671371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4a19-38aa-4659-8a6f-bdfd7bbdc107 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.676056 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.676639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.676916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.677184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.677826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.679465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.689108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1217/2461] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:44.693048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.703181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09f2df-c5f7-46d0-a3d0-ce753039323a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.707804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.708385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.708668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.708937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.709575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.711160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17c52f-13ff-45c0-a06f-118c51f3eb8d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.721901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1245/2462] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:44.725971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.736421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cef67-0813-45a6-8b5e-03ebb8eabe05 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.741025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.741658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.741938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.742245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] PATH_INFO: `/v3/projects/2c27998942ea4434bddd86fb2f7fb62a/users/b1f3fcb499cf44d1a11831a3c1985fdb/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.756698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c27998942ea4434bddd86fb2f7fb62a, user_id=b1f3fcb499cf44d1a11831a3c1985fdb, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.759367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.833848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1218/2463] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:44 2022] PUT /identity/v3/projects/2c27998942ea4434bddd86fb2f7fb62a/users/b1f3fcb499cf44d1a11831a3c1985fdb/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:44.842369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.842656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.842929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.870795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] MFA Rules not processed for user `b1f3fcb499cf44d1a11831a3c1985fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:44.871998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:44.919620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:44.936615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1246/2464] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:44.944388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.012250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf62ce-f6e3-4a2b-87ed-ff764c23d637 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.016943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.017549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.017827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.018134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.018764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.020667 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.114283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1219/2465] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:45.117496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.128123 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dae0c8-686d-4af9-a7fb-8213ca27883a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.187142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.187751 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.188087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.188369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.189020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.190982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.285091 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1247/2466] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:45 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:45.288964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.299608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcddf5b-1eeb-4dc4-b6ac-cf2d8a8b4ca6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.304236 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.304830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.305116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.305384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.306031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.307663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.319385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1220/2467] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:45.327912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.338282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7067e71-0a03-40ef-86af-fa502c75d51a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.342917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.343557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.343834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.344104 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] PATH_INFO: `/v3/projects/55e588d39e9345718a2027004d455e63/users/93a060ab15334042b81fb8b21a67973b/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.358598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e588d39e9345718a2027004d455e63, user_id=93a060ab15334042b81fb8b21a67973b, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.361227 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.484536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1248/2468] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:45 2022] PUT /identity/v3/projects/55e588d39e9345718a2027004d455e63/users/93a060ab15334042b81fb8b21a67973b/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:45.488560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.499668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8fc10-82aa-4c97-8d7b-84fd57e2dcd8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.510412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:45.560315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.560927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.561205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.561475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.562151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.563930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e51792-044f-4b1d-bb2f-b5fd725dbf07 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.574907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1221/2469] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:45.579387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.590530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5caa3-9113-4b5b-94d8-cb3a5583f48a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.595357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.596113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.596270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.596536 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] PATH_INFO: `/v3/projects/55e588d39e9345718a2027004d455e63/users/93a060ab15334042b81fb8b21a67973b/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.598885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e588d39e9345718a2027004d455e63, user_id=93a060ab15334042b81fb8b21a67973b, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.601480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.612323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1249/2470] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:54:45 2022] PUT /identity/v3/projects/55e588d39e9345718a2027004d455e63/users/93a060ab15334042b81fb8b21a67973b/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:45.628265 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5078e0-fc6f-4b85-9bfc-6b5b4b24e828 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.628553 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5078e0-fc6f-4b85-9bfc-6b5b4b24e828 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.628851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5078e0-fc6f-4b85-9bfc-6b5b4b24e828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.657214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0e5078e0-fc6f-4b85-9bfc-6b5b4b24e828 None None] MFA Rules not processed for user `93a060ab15334042b81fb8b21a67973b`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:45.658374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e5078e0-fc6f-4b85-9bfc-6b5b4b24e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:45.724827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1222/2471] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:54:45.735909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.737585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:45.795888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1450-bc18-4c76-8495-400c01f3e24b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.799496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.800094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.800370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.800639 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.853636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:45.856370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.857814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.866540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1250/2472] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:48.110279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:48.110790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:48.110887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:48.130551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:48.132532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:48.185971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1223/2473] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:54:48.337323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:48.338738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:48.409348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c72019-d7ce-4b9a-9701-84433d4ed15b service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:48.414451 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:48.415059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:48.415338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:48.415610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:48.423482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:48.425059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:48.432985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1251/2474] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:54:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:54:54.488467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:54.498636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe6df1-5fe2-4f75-a205-a3216e580330 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:54.502911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:54.503537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:54.503817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:54.504085 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:54.514392 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:54.515825 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3002d6-896a-4cc9-b529-964902d7c84a service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:54.525302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1224/2475] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:54:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:54:58.429844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=34 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:02.230320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:02.291665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7550-9b09-4a6f-abf0-88f2c435cd87 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:02.296696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:02.297309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:02.297587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:02.297859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] PATH_INFO: `/v3/users/b1f3fcb499cf44d1a11831a3c1985fdb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:02.299003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f3fcb499cf44d1a11831a3c1985fdb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:02.300884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:02.526846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f3fcb499cf44d1a11831a3c1985fdb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:02.538788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f3fcb499cf44d1a11831a3c1985fdb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:02.546374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f3fcb499cf44d1a11831a3c1985fdb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:02.547065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f3fcb499cf44d1a11831a3c1985fdb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:02.616551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f3fcb499cf44d1a11831a3c1985fdb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:02.622959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1252/2476] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:55:02 2022] DELETE /identity/v3/users/b1f3fcb499cf44d1a11831a3c1985fdb => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:02.630804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:02.630804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:02.630804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:02.647429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:02.648680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:02.667999 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:02.709731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1225/2477] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:02.719657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:02.721125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:02.777917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8593-d5bd-43e1-a56b-0af507bffa4a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:02.781454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:02.782045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:02.782354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:02.782626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:02.791629 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:02.793027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:02.803002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1253/2478] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:03.217823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:03.219389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:03.233912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc06b-ad45-479a-9b71-81eb7d62335f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:03.238866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:03.239502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:03.239788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:03.240207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] PATH_INFO: `/v3/projects/2c27998942ea4434bddd86fb2f7fb62a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:03.241343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c27998942ea4434bddd86fb2f7fb62a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:03.243299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:03.388397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c27998942ea4434bddd86fb2f7fb62a', 'request_id': 'req-dc0febf7-a414-42e0-8ded-199a07c4159d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.485310 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c27998942ea4434bddd86fb2f7fb62a', 'request_id': 'req-dc0febf7-a414-42e0-8ded-199a07c4159d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.491760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c27998942ea4434bddd86fb2f7fb62a', 'request_id': 'req-dc0febf7-a414-42e0-8ded-199a07c4159d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.503806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] The token cache is being invalidate because project 2c27998942ea4434bddd86fb2f7fb62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:55:03.504096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.504748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.506515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1226/2479] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:55:03 2022] DELETE /identity/v3/projects/2c27998942ea4434bddd86fb2f7fb62a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:03.510447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:03.520046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab1e0b-2a93-4680-affc-6d32b884bf95 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:03.539077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:03.576898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:03.577525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:03.577803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:03.578098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] PATH_INFO: `/v3/users/93a060ab15334042b81fb8b21a67973b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:03.579343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a060ab15334042b81fb8b21a67973b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:03.581319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:03.831898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a060ab15334042b81fb8b21a67973b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.844760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a060ab15334042b81fb8b21a67973b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.851740 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a060ab15334042b81fb8b21a67973b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.852457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a060ab15334042b81fb8b21a67973b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.936862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bdc55054-f50b-477c-9c33-d60c16ad7763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a060ab15334042b81fb8b21a67973b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:03.943137 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1254/2480] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:55:03 2022] DELETE /identity/v3/users/93a060ab15334042b81fb8b21a67973b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:04.356996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.359614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:04.382463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0febf7-a414-42e0-8ded-199a07c4159d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.440984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.441745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.442028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.442329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] PATH_INFO: `/v3/projects/55e588d39e9345718a2027004d455e63` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.443466 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e588d39e9345718a2027004d455e63)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.445623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:04.612377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e588d39e9345718a2027004d455e63', 'request_id': 'req-9d41b9be-321b-4b7e-a544-235366cb23fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:04.614906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:04.742419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e588d39e9345718a2027004d455e63', 'request_id': 'req-9d41b9be-321b-4b7e-a544-235366cb23fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:04.749697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e588d39e9345718a2027004d455e63', 'request_id': 'req-9d41b9be-321b-4b7e-a544-235366cb23fe'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:04.761403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] The token cache is being invalidate because project 55e588d39e9345718a2027004d455e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:55:04.761695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:04.762362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9d41b9be-321b-4b7e-a544-235366cb23fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:04.763868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1227/2481] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:55:04 2022] DELETE /identity/v3/projects/55e588d39e9345718a2027004d455e63 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:04.769744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.770371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.770371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.791082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:04.792338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:04.854446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1255/2482] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:04.863558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.863848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.864121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.884702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:04.886443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:04.894345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1228/2483] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:04.898816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.908452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bf804-a407-4a20-8f0f-bf6905102581 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.912963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.913549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.913822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.914110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.914748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:04.915045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.916634 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:04.928430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1256/2484] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:04.932828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.943083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c5647-cd52-4dcd-a98c-80420182245a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.947887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.948579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.948765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.949035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.949680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.951907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.068594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1229/2485] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:55:04 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:55:05.072956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:05.083896 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a4243-0b84-4a5f-8a3d-25c5d55b113c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:05.112566 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:05.147942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:05.149314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.149314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.149314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.149782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:05.151747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.230158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1257/2486] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:55:05 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:55:05.231535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:05.242731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a12a2-0b95-4180-a1df-6200c5244efe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:05.247795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:05.248458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.248753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.249025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.249735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:05.251404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.261953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1230/2487] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:55:05.266032 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:05.276130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2c84a-660e-4785-b72d-df7cb3027a47 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:05.281437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:05.282107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.282392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.282662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.283304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:05.284893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de3ea0-3bb2-4edc-9693-3fbb3fe6a8da admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.294756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1258/2488] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:55:05.299044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:05.309143 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294bb00-e241-4a46-a2c5-69ddf98348eb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:05.314744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:05.315429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.315842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.316126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] PATH_INFO: `/v3/projects/0ac3613cfe9a4d2f9bccc136570e9fa6/users/5d24376b9fbd43b6b474c9547fd33313/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.332496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac3613cfe9a4d2f9bccc136570e9fa6, user_id=5d24376b9fbd43b6b474c9547fd33313, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:05.335327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.404728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1231/2489] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:55:05 2022] PUT /identity/v3/projects/0ac3613cfe9a4d2f9bccc136570e9fa6/users/5d24376b9fbd43b6b474c9547fd33313/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:05.415993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284aae-7d91-4265-a47f-c722dc89e6c6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.416281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284aae-7d91-4265-a47f-c722dc89e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.416558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284aae-7d91-4265-a47f-c722dc89e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.449924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bc284aae-7d91-4265-a47f-c722dc89e6c6 None None] MFA Rules not processed for user `5d24376b9fbd43b6b474c9547fd33313`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:05.451205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bc284aae-7d91-4265-a47f-c722dc89e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:05.518232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1259/2490] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:05.529727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:05.531502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:05.591044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8f46a-8782-4dd6-99f4-2ba142c1e399 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:05.597299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:05.597299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:05.597299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:05.597299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:05.604734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:05.606206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503b791-3191-4083-99f9-c8bf66eb85a6 service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:05.616242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1232/2491] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:06.886923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270548-dc51-406f-9762-f8e774124230 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:06.887453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270548-dc51-406f-9762-f8e774124230 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:06.887453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5270548-dc51-406f-9762-f8e774124230 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:06.889061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1260/2492] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:06.894916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2ae8-6ca2-4a7c-933a-4e38b220ac1b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:06.895387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2ae8-6ca2-4a7c-933a-4e38b220ac1b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:06.895487 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d2ae8-6ca2-4a7c-933a-4e38b220ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:06.928822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ff3d2ae8-6ca2-4a7c-933a-4e38b220ac1b None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:06.930072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ff3d2ae8-6ca2-4a7c-933a-4e38b220ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:06.987683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:07.007430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1233/2493] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:55:07.019773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5270548-dc51-406f-9762-f8e774124230 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:07.021431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f5270548-dc51-406f-9762-f8e774124230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:07.035532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5270548-dc51-406f-9762-f8e774124230 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:07.039153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:07.039747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.040026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.040301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.050877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:07.052345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bb29e-005e-47d7-b9c3-107f1ed37295 service glance] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:07.063539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1261/2494] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:07.361780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.362080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.362461 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.364005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1234/2495] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:07.368867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5594-d171-4da6-829a-bbe019d2c4e0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.369148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5594-d171-4da6-829a-bbe019d2c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.369419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5594-d171-4da6-829a-bbe019d2c4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.391173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-276b5594-d171-4da6-829a-bbe019d2c4e0 None None] MFA Rules not processed for user `b062fa227cd043719c7eff9d2b7e4bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:07.392446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-276b5594-d171-4da6-829a-bbe019d2c4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:07.447860 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1262/2496] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:55:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:07.460686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:07.462641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:07.476306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb0d85-bd7b-4ae8-a781-39f631447c7d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:07.480407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:07.481012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.481312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.481636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.489691 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:07.491200 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2e1bd-eebd-431d-86a7-c58a5c8d4750 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:07.499898 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1235/2497] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:07.585979 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f031a6-60b1-41a5-96c7-ec7f6eafd058 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.586347 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f031a6-60b1-41a5-96c7-ec7f6eafd058 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.586626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f031a6-60b1-41a5-96c7-ec7f6eafd058 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.587966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1263/2498] 10.222.0.60 () {54 vars in 921 bytes} [Fri Sep 9 11:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:07.593623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eee24-91be-494f-b7d6-80f452c15dc9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.599633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eee24-91be-494f-b7d6-80f452c15dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.599633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eee24-91be-494f-b7d6-80f452c15dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.614068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-437eee24-91be-494f-b7d6-80f452c15dc9 None None] MFA Rules not processed for user `e7b89441837347cba3200f9104c0f16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:07.615543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-437eee24-91be-494f-b7d6-80f452c15dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:07.627891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1236/2499] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Sep 9 11:55:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:07.638401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f031a6-60b1-41a5-96c7-ec7f6eafd058 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:07.650442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f031a6-60b1-41a5-96c7-ec7f6eafd058 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:07.654618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:07.655263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.655545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.655818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.666432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:07.667898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:07.679068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1264/2500] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:10.245903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:10.246390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:10.246621 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:10.266280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:10.267544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:10.326229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1237/2501] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Sep 9 11:55:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:10.338848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:10.340352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:10.365404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:10.418779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25784a9-85e0-45da-8377-e2e76be5787e service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:10.424621 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:10.425432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:10.425718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:10.425994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:10.436022 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:10.436839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896cda9-7b9b-464c-9b21-268992dfbd36 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:10.449970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1265/2502] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:55:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:55:20.319473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=38 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:20.514607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=23 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:34.532353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:34.599778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75833408-2777-4bdb-b7e7-c94d08d19c9a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:34.604670 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:34.605298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:34.605576 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:34.605865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] PATH_INFO: `/v3/users/5d24376b9fbd43b6b474c9547fd33313` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:34.607027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d24376b9fbd43b6b474c9547fd33313)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:34.608952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:34.809084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d24376b9fbd43b6b474c9547fd33313'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:34.821690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d24376b9fbd43b6b474c9547fd33313'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:34.829541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d24376b9fbd43b6b474c9547fd33313'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:34.893043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d24376b9fbd43b6b474c9547fd33313'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:34.893669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d24376b9fbd43b6b474c9547fd33313'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:34.900078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1238/2503] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:55:34 2022] DELETE /identity/v3/users/5d24376b9fbd43b6b474c9547fd33313 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:34.908874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:34.908874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:34.908874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:34.924743 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:34.925994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:34.986899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1266/2504] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:34.997535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:34.999058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:35.057944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51986df-8135-4f16-810d-106e6f8d72bf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:35.062013 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:35.062586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:35.062863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:35.063136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:35.072201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:35.073622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b857089-2215-4e01-8115-da4710e40345 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:35.083877 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1239/2505] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:35.591887 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:35.592720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:35.607270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77938d28-f152-44d2-ad1a-d7f761817bc4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:35.613523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:35.614825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:35.615268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:35.615375 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] PATH_INFO: `/v3/projects/0ac3613cfe9a4d2f9bccc136570e9fa6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:35.617987 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac3613cfe9a4d2f9bccc136570e9fa6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:35.620043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:35.749745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac3613cfe9a4d2f9bccc136570e9fa6', 'request_id': 'req-d76a2110-09ad-4927-a901-89301eed69a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:35.756379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac3613cfe9a4d2f9bccc136570e9fa6', 'request_id': 'req-d76a2110-09ad-4927-a901-89301eed69a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:35.763108 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac3613cfe9a4d2f9bccc136570e9fa6', 'request_id': 'req-d76a2110-09ad-4927-a901-89301eed69a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:35.840470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] The token cache is being invalidate because project 0ac3613cfe9a4d2f9bccc136570e9fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:55:35.840769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:35.841492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d76a2110-09ad-4927-a901-89301eed69a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:55:35.843019 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1267/2506] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:55:35 2022] DELETE /identity/v3/projects/0ac3613cfe9a4d2f9bccc136570e9fa6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:35.848011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:35.850009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:35.850009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:35.868686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:35.869900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:35.887304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:35.929508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1240/2507] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:35.937391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11823-55ed-421b-a722-11570846097d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:35.937919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11823-55ed-421b-a722-11570846097d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:35.937919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b11823-55ed-421b-a722-11570846097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:35.958288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-07b11823-55ed-421b-a722-11570846097d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:35.959494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-07b11823-55ed-421b-a722-11570846097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:35.968046 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1268/2508] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:35.972167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:35.981726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129f63e-f533-4a58-8fb4-cc6c57e071b9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:35.986253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:35.986846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:35.987124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:35.987393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:35.988034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:35.988360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:35.989942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.001348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1241/2509] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:36.005498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b11823-55ed-421b-a722-11570846097d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.015246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b11823-55ed-421b-a722-11570846097d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.019881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.020503 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.020785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.021052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.021692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.024216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.123099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1269/2510] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:55:36 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:55:36.127467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.138342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46085-56b0-4238-b955-36509475304d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.193470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.194101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.194388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.194664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.195406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.197945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.274473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1242/2511] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:55:36 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:55:36.278590 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.288795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9468c5-2117-4b6e-a670-9e26896d2798 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.293560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.294194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.294474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.294747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.295396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.296963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.307163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1270/2512] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:55:36.313000 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.323777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db0d3e-9e64-428f-92c2-0ff8cd13e824 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.328333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.328916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.329455 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.329722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.330398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.331997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36992ce-6163-49fc-aae7-52986cf33984 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.342979 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1243/2513] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:55:36.347084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.357176 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be836-c3f0-4221-8d00-c6375405af98 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.361913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.370124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.370124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.370124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] PATH_INFO: `/v3/projects/3f81b5c49bf94b0194786b77a6498ab2/users/beec2ca2beb240bdba65cef954788410/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.378856 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f81b5c49bf94b0194786b77a6498ab2, user_id=beec2ca2beb240bdba65cef954788410, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.382025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.527525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1271/2514] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:55:36 2022] PUT /identity/v3/projects/3f81b5c49bf94b0194786b77a6498ab2/users/beec2ca2beb240bdba65cef954788410/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:36.538266 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9f7e2-dea2-4caa-9bc2-10df846f98a3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.538663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9f7e2-dea2-4caa-9bc2-10df846f98a3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.539030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9f7e2-dea2-4caa-9bc2-10df846f98a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.567407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3af9f7e2-dea2-4caa-9bc2-10df846f98a3 None None] MFA Rules not processed for user `beec2ca2beb240bdba65cef954788410`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:36.568614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3af9f7e2-dea2-4caa-9bc2-10df846f98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:36.632390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1244/2515] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:36.643977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:36.645778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:36.659125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:36.702528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102ba48-230c-4ab6-b021-4f657ec48ce8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:36.707507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:36.713197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:36.713197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:36.713197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:36.717599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:36.719118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:36.728258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1272/2516] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:37.303147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:37.307766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:37.307766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:37.324617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] MFA Rules not processed for user `295dd5a7b7774239bf95df04555611f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:37.325830 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:37.337927 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:37.388943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1245/2517] 10.222.0.60 () {58 vars in 967 bytes} [Fri Sep 9 11:55:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:55:37.401504 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:37.402952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:37.473854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ff2f9-36d7-4c85-935e-bc416a90632a service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:37.479512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da9b976e48d3468b82d9168efced0b68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:37.479997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:37.480279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:37.480554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:37.489036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:37.490799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:37.500780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1273/2518] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Sep 9 11:55:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:55:38.123710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:38.125316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:38.184328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615ad49-70b4-4848-b641-5dbf9d534e92 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:38.187905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:38.188504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:38.188851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:38.189118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:38.190998 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:38.268894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:38.270352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:38.281702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1246/2519] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:39.471459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:39.472889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:39.485066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfcf4-c85e-4204-833d-e7b58fa8b181 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:39.488830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:39.489428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:39.489703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:39.489971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:39.491703 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:39.505674 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:39.507125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb28b58-3323-4673-90b8-c0f2a719554f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:39.522408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1274/2520] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:55:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:55:48.276873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=71 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:49.515358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=59 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:02.050271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:02.124749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dec0ae-b922-415c-82a5-5231578ef137 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:02.134297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:02.135501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:02.135501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:02.142165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] PATH_INFO: `/v3/users/beec2ca2beb240bdba65cef954788410` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:02.144090 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beec2ca2beb240bdba65cef954788410)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:02.146174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:02.405409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beec2ca2beb240bdba65cef954788410'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:02.417369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beec2ca2beb240bdba65cef954788410'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:02.424562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beec2ca2beb240bdba65cef954788410'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:02.482594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beec2ca2beb240bdba65cef954788410'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:02.483204 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beec2ca2beb240bdba65cef954788410'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:02.489179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1247/2521] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:02 2022] DELETE /identity/v3/users/beec2ca2beb240bdba65cef954788410 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:02.493619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:02.494098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:02.494200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:02.513416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:02.515248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:02.573077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1275/2522] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:02.585434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:02.586912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:02.643305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d94fa-7c1f-475f-8454-30d1403d0772 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:02.646799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:02.647405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:02.647684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:02.647956 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:02.656802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:02.658294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded01ba-b7cb-443f-9232-abb61fed2ca0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:02.667525 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1248/2523] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:03.095965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:03.097514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:03.111426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02810ae-2ebe-42de-b469-a524f7454729 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:03.116353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:03.116980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:03.117258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:03.117529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] PATH_INFO: `/v3/projects/3f81b5c49bf94b0194786b77a6498ab2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:03.118669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f81b5c49bf94b0194786b77a6498ab2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:03.120564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:03.248258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f81b5c49bf94b0194786b77a6498ab2', 'request_id': 'req-bf4cd036-c66d-4e74-bf6b-f02fe896830e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:03.254678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f81b5c49bf94b0194786b77a6498ab2', 'request_id': 'req-bf4cd036-c66d-4e74-bf6b-f02fe896830e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:03.260150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f81b5c49bf94b0194786b77a6498ab2', 'request_id': 'req-bf4cd036-c66d-4e74-bf6b-f02fe896830e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:03.338560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] The token cache is being invalidate because project 3f81b5c49bf94b0194786b77a6498ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:03.338851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:03.339470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:03.340988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1276/2524] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:03 2022] DELETE /identity/v3/projects/3f81b5c49bf94b0194786b77a6498ab2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:06.775043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:06.775561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:06.775663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:06.794664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] MFA Rules not processed for user `b193d0924fb448bfac2027034cb8dfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:06.795867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:06.812929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:06.851294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1249/2525] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:07.159732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:07.161159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:07.216749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cd036-c66d-4e74-bf6b-f02fe896830e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:07.221189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:07.221790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:07.222092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:07.222368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:07.224073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:07.233433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:07.234912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:07.242866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1277/2526] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:07.629277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:07.630762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:07.707994 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f4c3e-d734-4ea7-b0ab-18e9b840d6bc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:07.713098 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:07.713737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:07.714019 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:07.714332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:07.716066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:07.728378 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:07.729997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:07.740735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1250/2527] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:11.459228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:11.460877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:11.471323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918caae-aab9-4cee-b8b0-93e0ac6c0665 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:11.475913 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:11.476738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:11.477118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:11.477558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:11.479381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:11.489391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:11.490845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:11.499100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1278/2528] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:12.420302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:12.421946 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:12.500402 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dff60-359d-4a76-8080-4b6b31fc5432 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:12.506150 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:12.506875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:12.507160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:12.507503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:12.509257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:12.522540 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:12.524152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:12.535410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1251/2529] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:21.495384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:22.530311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=81 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.571808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:25.580312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fe38-cbc4-497a-aa61-5c5ca7307c9a service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.584209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.584797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.585076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.585349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.587101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:25.649995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.651379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.660935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1279/2530] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:29.056222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.057899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:29.072113 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a15a4-f65c-422c-94fb-d33bd328eab7 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.076693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.077263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.077539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.077807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] PATH_INFO: `/v3/users/683fe1045ddb4bc1a8a94d3948495f96` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.092413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=683fe1045ddb4bc1a8a94d3948495f96)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.094371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.251862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683fe1045ddb4bc1a8a94d3948495f96'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.264650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683fe1045ddb4bc1a8a94d3948495f96'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.272409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683fe1045ddb4bc1a8a94d3948495f96'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.363917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683fe1045ddb4bc1a8a94d3948495f96'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.364958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1f712483-dd77-47b6-ace2-d170b5914e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683fe1045ddb4bc1a8a94d3948495f96'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.373036 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1252/2531] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:29 2022] DELETE /identity/v3/users/683fe1045ddb4bc1a8a94d3948495f96 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:29.605427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.616511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472ab9-c054-47d8-b71d-df242ce95593 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.675771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.676495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.676814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.677170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] PATH_INFO: `/v3/projects/9ac7346b9af144f48f0760796c594016` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.688114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac7346b9af144f48f0760796c594016)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.689998 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.828004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac7346b9af144f48f0760796c594016', 'request_id': 'req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.835205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac7346b9af144f48f0760796c594016', 'request_id': 'req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.840885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac7346b9af144f48f0760796c594016', 'request_id': 'req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.935068 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] The token cache is being invalidate because project 9ac7346b9af144f48f0760796c594016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:29.935291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.935949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3f1475d6-b669-4ade-928c-0a8cc89cb9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:29.937464 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1280/2532] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:29 2022] DELETE /identity/v3/projects/9ac7346b9af144f48f0760796c594016 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:29.942433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.943218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.943218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.962653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:29.963892 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:30.021749 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1253/2533] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:30.030789 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.031078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.031369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.051129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:30.052343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:30.061191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1281/2534] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:30.065169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.075198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aee63-2de8-49bf-96f3-ca7397c7f3b7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.079951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.080597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.080872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.081141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.081781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:30.082106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.084043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.095816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1254/2535] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:30.100679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.109949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d01d3-5450-43c5-b5ac-fcaa341d7990 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.114649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.119326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.119622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.119900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.120555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.123343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.130886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:30.220971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1282/2536] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:30.222105 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.234809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0634a-cc13-4627-95ae-27cc3743dc04 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.281884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:30.294357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.294968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.295253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.295546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.296224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.298401 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.370387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1255/2537] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:30 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:30.374625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.385207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd772-ab41-4291-8ae0-c7727dcf423d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.390302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.394729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.394729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.394729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.394729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.394729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.404457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1283/2538] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:30.408542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.419673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02265-6ea0-4e3e-a07c-91183ff29da8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.424451 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.425041 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.425382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.425593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.426259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.427886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30a619-6a45-48b1-b99b-27e3cd6ebac8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.438486 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1256/2539] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:30.448798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.452985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ab5c-8444-4ffe-aba4-e64a6c0c91dd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.458134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.459269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.459554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.459828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] PATH_INFO: `/v3/projects/ecf00e4cac9f4ec1ad856d1778ab6e94/users/98213a8bd10940a1b8d85347322cc8d4/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.476459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf00e4cac9f4ec1ad856d1778ab6e94, user_id=98213a8bd10940a1b8d85347322cc8d4, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.479153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.639063 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1284/2540] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:30 2022] PUT /identity/v3/projects/ecf00e4cac9f4ec1ad856d1778ab6e94/users/98213a8bd10940a1b8d85347322cc8d4/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:30.650493 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.650805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.651084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.678161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] MFA Rules not processed for user `98213a8bd10940a1b8d85347322cc8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:30.679371 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:30.741636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1257/2541] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:30.746306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.926855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49141d-000d-444a-8063-43c732c97869 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.932090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.932678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.932950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.933214 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.933838 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.935708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.033547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1285/2542] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:31.038296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.049194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729f4df-5f1b-429f-8a2c-8f5e847e6704 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.105308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.106006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.106278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.106545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.107168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.109101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.203228 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1258/2543] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:31 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:31.207197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.218195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9460a-685e-4d6c-982b-7926cc050221 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.222951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.223560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.223833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.224105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.225364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.227173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.237567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1286/2544] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:31.241692 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.251833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b3fae-4bf4-4286-91e4-d8a6dca61c50 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.256911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.257564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.257844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.258147 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] PATH_INFO: `/v3/projects/ef640ef3f91249d8ae9fa50540a4a242/users/f4bfa6d01aa84f9d8afbc29983235d4c/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.273473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef640ef3f91249d8ae9fa50540a4a242, user_id=f4bfa6d01aa84f9d8afbc29983235d4c, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.276203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.358029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1259/2545] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:31 2022] PUT /identity/v3/projects/ef640ef3f91249d8ae9fa50540a4a242/users/f4bfa6d01aa84f9d8afbc29983235d4c/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:31.362408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.372448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3b2d-89b1-4db0-bdb7-7b6800c8afd5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.428297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.428897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.429174 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.429444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.430114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.431683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a037e-fe69-4b29-8def-ecade3ce3fe7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.441082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1287/2546] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:31.447570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.456434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fde4a-331b-49a0-bf73-7db9251fc85e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.461971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.462648 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.462929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.463201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] PATH_INFO: `/v3/projects/ef640ef3f91249d8ae9fa50540a4a242/users/f4bfa6d01aa84f9d8afbc29983235d4c/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.465114 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef640ef3f91249d8ae9fa50540a4a242, user_id=f4bfa6d01aa84f9d8afbc29983235d4c, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.467873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.477964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1260/2547] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:31 2022] PUT /identity/v3/projects/ef640ef3f91249d8ae9fa50540a4a242/users/f4bfa6d01aa84f9d8afbc29983235d4c/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:31.488377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.488652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.488946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.516945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] MFA Rules not processed for user `f4bfa6d01aa84f9d8afbc29983235d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:31.518181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:31.528098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:31.585814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1288/2548] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:31.597356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.599110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:31.610177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:31.655633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef9af-66ff-42c8-9d40-253f8035b571 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.659256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.659868 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.660142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.660408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.669532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.671002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.680263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1261/2549] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:31.874510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:31.884475 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46cfec-789a-4f82-9a7c-5fe280bc74a5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:31.888593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:31.889177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:31.889452 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:31.889716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:31.943251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:31.945266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:31.953786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1289/2550] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:41.701191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=22 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:41.950168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=52 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.619710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.628457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8473c40-3e2b-406b-a2b0-ee6723f68e3a service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.632852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.633419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.633729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.634003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.636103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:44.716479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.717913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493cb16-20f1-459c-955b-663593526b0b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.736603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1262/2551] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:44.927495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.990524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099a3db-b688-4704-929f-cb88c8c213f5 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.995099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.995738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.996011 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.996278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] PATH_INFO: `/v3/users/98213a8bd10940a1b8d85347322cc8d4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.997409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98213a8bd10940a1b8d85347322cc8d4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.999308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:45.253872 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98213a8bd10940a1b8d85347322cc8d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:45.266543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98213a8bd10940a1b8d85347322cc8d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:45.273312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98213a8bd10940a1b8d85347322cc8d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:45.274118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98213a8bd10940a1b8d85347322cc8d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:45.405045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98213a8bd10940a1b8d85347322cc8d4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:45.412397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1290/2552] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:44 2022] DELETE /identity/v3/users/98213a8bd10940a1b8d85347322cc8d4 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:45.416850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:45.417141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:45.417412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:45.437368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:45.438623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:45.503279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1263/2553] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:45.513770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:45.515198 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:45.574643 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7bba-913e-49de-ab8b-e1f22f8d3e79 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:45.578294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:45.578898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:45.579175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:45.579445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:45.590178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:45.592552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:45.603119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1291/2554] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:46.017566 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:46.021791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:46.031776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3ee7-f5dd-4807-a05c-a879464ffe08 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:46.037319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:46.037957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:46.038273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:46.038542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] PATH_INFO: `/v3/projects/ecf00e4cac9f4ec1ad856d1778ab6e94` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:46.039676 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf00e4cac9f4ec1ad856d1778ab6e94)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:46.042181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:46.182608 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf00e4cac9f4ec1ad856d1778ab6e94', 'request_id': 'req-18633772-79a7-4954-8d6f-f551161d40b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.267158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf00e4cac9f4ec1ad856d1778ab6e94', 'request_id': 'req-18633772-79a7-4954-8d6f-f551161d40b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.273042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf00e4cac9f4ec1ad856d1778ab6e94', 'request_id': 'req-18633772-79a7-4954-8d6f-f551161d40b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.274720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:46.285330 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] The token cache is being invalidate because project ecf00e4cac9f4ec1ad856d1778ab6e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:46.285562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.286258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.287751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1264/2555] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:46 2022] DELETE /identity/v3/projects/ecf00e4cac9f4ec1ad856d1778ab6e94 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:46.292496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:46.302005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a171492-62fb-4789-8b8f-c70814c17c5e service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:46.320318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:46.361814 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:46.362478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:46.362755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:46.363025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] PATH_INFO: `/v3/users/f4bfa6d01aa84f9d8afbc29983235d4c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:46.364342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4bfa6d01aa84f9d8afbc29983235d4c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:46.366294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:46.584130 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4bfa6d01aa84f9d8afbc29983235d4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.596477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4bfa6d01aa84f9d8afbc29983235d4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.603294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4bfa6d01aa84f9d8afbc29983235d4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.603999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4bfa6d01aa84f9d8afbc29983235d4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.679288 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7d609035-98fe-4184-aa9a-bcaed06f3ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4bfa6d01aa84f9d8afbc29983235d4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:46.686146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1292/2556] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:46 2022] DELETE /identity/v3/users/f4bfa6d01aa84f9d8afbc29983235d4c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:47.087707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.089299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:47.103475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18633772-79a7-4954-8d6f-f551161d40b1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.161136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.161761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.162039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.162337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] PATH_INFO: `/v3/projects/ef640ef3f91249d8ae9fa50540a4a242` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.163439 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef640ef3f91249d8ae9fa50540a4a242)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.165887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.278822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef640ef3f91249d8ae9fa50540a4a242', 'request_id': 'req-96c1012d-38b0-4a28-9217-6bab1f134415'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:47.354617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef640ef3f91249d8ae9fa50540a4a242', 'request_id': 'req-96c1012d-38b0-4a28-9217-6bab1f134415'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:47.367565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef640ef3f91249d8ae9fa50540a4a242', 'request_id': 'req-96c1012d-38b0-4a28-9217-6bab1f134415'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:47.387210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] The token cache is being invalidate because project ef640ef3f91249d8ae9fa50540a4a242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:47.387508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:47.388167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-96c1012d-38b0-4a28-9217-6bab1f134415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:47.389675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1265/2557] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:47 2022] DELETE /identity/v3/projects/ef640ef3f91249d8ae9fa50540a4a242 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:47.402481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.403079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.403079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.424038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:47.425295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:47.491285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1293/2558] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:47.499364 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.499741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.500025 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.520838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:47.522148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:47.530761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1266/2559] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:47.535706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.546521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fca82-ef9e-4c1e-ae5c-c919628f2b0e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.551206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.551945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.552287 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.552627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.553357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:47.553729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.556036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.570096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1294/2560] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:47.573871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.584312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb13adf-03d1-4b33-8430-f59039091628 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.589336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.590092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.590298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.590569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.591198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.593107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.700942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1267/2561] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:47 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:47.703117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.714605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c7076-7c9c-4d6a-831f-e9c556692489 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.742318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:47.771993 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.772610 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.772867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.773141 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.773769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.776254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.862144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1295/2562] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:47 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:47.866303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.877863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6c68a-ca25-4a67-bd76-92b7213fd772 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.883165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.884710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.885132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.885414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.886688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.888092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.906368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1268/2563] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:47.907698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.918043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5040-7f39-4b8b-ac90-7042782dde7f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.922982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.923676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.924018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.924353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.925062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.926772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b7ad0-67dc-48b4-b571-c63cd7375022 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.939949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1296/2564] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:47.941375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.952552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d73be7-7ed4-4557-b96e-3d7d95d6c585 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.957423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.958100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.958382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.958679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] PATH_INFO: `/v3/projects/712b24090d1f41aaa1aa57bc5a62c519/users/80b6c0332b584dcfbb41b1c6b0de26f6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.979932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712b24090d1f41aaa1aa57bc5a62c519, user_id=80b6c0332b584dcfbb41b1c6b0de26f6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.982641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.080350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1269/2565] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:47 2022] PUT /identity/v3/projects/712b24090d1f41aaa1aa57bc5a62c519/users/80b6c0332b584dcfbb41b1c6b0de26f6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:48.091676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.091954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.092263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.120045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] MFA Rules not processed for user `80b6c0332b584dcfbb41b1c6b0de26f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:48.121165 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:48.195541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1297/2566] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:48.199304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.222620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.264107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5126c2-30b8-4777-8752-91055e3e5229 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.268732 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.269331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.269606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.269873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.270523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.272420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.410035 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1270/2567] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:48 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:48.411359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.422474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79254ab9-34e8-4aba-a838-7423213d07d9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.503178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.503863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.504207 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.504544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.505294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.507333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.617463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1298/2568] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:48 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:48.621934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.634162 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a49e64-a012-4d92-9b03-68f28f7bc807 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.645307 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.645943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.646278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.646616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.647203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.648831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.660473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1271/2569] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:48.665407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.676180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7d99b-b90e-4d9c-be2d-996440fdfc82 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.681926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.682627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.682904 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.683183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] PATH_INFO: `/v3/projects/84b2e0f552854de59268b091871a34e4/users/7db25287f0fe49038090d65c68a34e7b/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.700462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b2e0f552854de59268b091871a34e4, user_id=7db25287f0fe49038090d65c68a34e7b, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.703170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.799879 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1299/2570] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:48 2022] PUT /identity/v3/projects/84b2e0f552854de59268b091871a34e4/users/7db25287f0fe49038090d65c68a34e7b/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:48.803874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.815137 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2a2c9-3098-4958-a640-e3caab4cf35c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.873158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.873904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.874389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.874738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.875452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.877070 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a9130-1fa9-4a16-9d0c-6cda2e07afcb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.887453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1272/2571] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:48.891706 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.897353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.903397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffe12e-1902-4ccb-a918-d4a592367fcd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.908473 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.909143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.909422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.909696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] PATH_INFO: `/v3/projects/84b2e0f552854de59268b091871a34e4/users/7db25287f0fe49038090d65c68a34e7b/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.911694 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b2e0f552854de59268b091871a34e4, user_id=7db25287f0fe49038090d65c68a34e7b, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.914635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.925988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1300/2572] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:48 2022] PUT /identity/v3/projects/84b2e0f552854de59268b091871a34e4/users/7db25287f0fe49038090d65c68a34e7b/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:48.937764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.938209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.938516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.971594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] MFA Rules not processed for user `7db25287f0fe49038090d65c68a34e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:48.972850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:49.060817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1273/2573] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:49.071929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.073698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:49.137716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc3b86-7422-4f1b-bcc3-dd6a06b90b97 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.141635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.142282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.142563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.142837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.153416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.154866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed71f55-3acc-4e48-95ad-88ccf30c4c8a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.165874 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1301/2574] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:49.309342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.331800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:49.382808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bbd28-bfe1-46ec-8b96-16535ec7724e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.389672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.389672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.389672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.389672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] PATH_INFO: `/v3/users/80b6c0332b584dcfbb41b1c6b0de26f6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.390112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b6c0332b584dcfbb41b1c6b0de26f6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.391936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.595146 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b6c0332b584dcfbb41b1c6b0de26f6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:49.609327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b6c0332b584dcfbb41b1c6b0de26f6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:49.617069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b6c0332b584dcfbb41b1c6b0de26f6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:49.719331 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b6c0332b584dcfbb41b1c6b0de26f6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:49.719930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b6c0332b584dcfbb41b1c6b0de26f6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:49.726568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1274/2575] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:49 2022] DELETE /identity/v3/users/80b6c0332b584dcfbb41b1c6b0de26f6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:49.733779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.734327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.734436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.754394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:49.755853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:49.817916 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1302/2576] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:49.829770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.831277 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:49.890131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1bb17-33db-4678-ba36-3b98c004d818 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.893690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.894325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.894603 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.894870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.904524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.906046 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.916386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1275/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:50.344251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:50.345797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:50.363156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecf715-6d40-4a1a-ba6e-841a66994a48 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:50.369834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:50.371006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:50.379248 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:50.380505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] PATH_INFO: `/v3/projects/712b24090d1f41aaa1aa57bc5a62c519` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:50.381553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712b24090d1f41aaa1aa57bc5a62c519)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:50.382963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:50.543192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712b24090d1f41aaa1aa57bc5a62c519', 'request_id': 'req-7bf69311-588f-496b-94a7-8643dae1af93'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.550114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712b24090d1f41aaa1aa57bc5a62c519', 'request_id': 'req-7bf69311-588f-496b-94a7-8643dae1af93'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.556556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712b24090d1f41aaa1aa57bc5a62c519', 'request_id': 'req-7bf69311-588f-496b-94a7-8643dae1af93'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.639074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] The token cache is being invalidate because project 712b24090d1f41aaa1aa57bc5a62c519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:50.639372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.640025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.641587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1303/2578] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:50 2022] DELETE /identity/v3/projects/712b24090d1f41aaa1aa57bc5a62c519 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:50.645908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:50.655751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766699f-02ad-4b3b-b532-9ab931cb3cc2 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:50.691984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:50.715630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:50.716261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:50.716537 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:50.716808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] PATH_INFO: `/v3/users/7db25287f0fe49038090d65c68a34e7b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:50.718181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db25287f0fe49038090d65c68a34e7b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:50.720096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:50.910377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db25287f0fe49038090d65c68a34e7b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.923219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db25287f0fe49038090d65c68a34e7b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:50.931152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db25287f0fe49038090d65c68a34e7b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.005180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db25287f0fe49038090d65c68a34e7b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.005800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2a2d34-1b83-431d-876c-9f697ef1b6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db25287f0fe49038090d65c68a34e7b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.012529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1276/2579] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:50 2022] DELETE /identity/v3/users/7db25287f0fe49038090d65c68a34e7b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:51.424410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:51.426031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:51.438865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69311-588f-496b-94a7-8643dae1af93 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:51.450767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:51.496769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:51.497394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.497672 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.497942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] PATH_INFO: `/v3/projects/84b2e0f552854de59268b091871a34e4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.499080 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b2e0f552854de59268b091871a34e4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:51.501016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:51.645418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b2e0f552854de59268b091871a34e4', 'request_id': 'req-27839734-ea78-43ed-8d04-2e5fbfb2115d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.652893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b2e0f552854de59268b091871a34e4', 'request_id': 'req-27839734-ea78-43ed-8d04-2e5fbfb2115d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.659105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b2e0f552854de59268b091871a34e4', 'request_id': 'req-27839734-ea78-43ed-8d04-2e5fbfb2115d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.816012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] The token cache is being invalidate because project 84b2e0f552854de59268b091871a34e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:51.816308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.816942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-27839734-ea78-43ed-8d04-2e5fbfb2115d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:51.818457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1304/2580] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:51 2022] DELETE /identity/v3/projects/84b2e0f552854de59268b091871a34e4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:51.823901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.824379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.824479 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.844801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:51.846014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:51.904722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1277/2581] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:51.912445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.912735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.913035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.933003 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:51.934245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:51.943808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1305/2582] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:51.948622 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:51.958390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b0beb-5c43-417b-98a1-5a345bf1a7bf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:51.965107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:51.965107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.965107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.965107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.965542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:51.965542 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:51.966881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:51.978427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1278/2583] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:51.982758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:51.992075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0e1b-523c-47a6-94b5-10a6c5730f88 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:51.998680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:51.998680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.998680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.998680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.999129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.000807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.080411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1306/2584] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:51 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:52.084955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.095218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a4b38-2e9c-48f7-9709-bd5a0d94024d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.137034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:52.151417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.152069 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.152349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.152614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.153252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.155270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.256293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1279/2585] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:52 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:52.261028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.271273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c84e62-83e2-4284-a084-e1dad6b089c2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.275841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.276425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.276697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.276963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.277769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.279364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.288925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1307/2586] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:52.293449 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.306470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa79b-44c4-461a-99ac-611a050144e5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.311099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.311711 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.311989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.312257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.312896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.314504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d175f33-b1a9-402c-bbd5-9f82de28ffa6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.325171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1280/2587] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:52.329659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.343705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64731cb-b1a9-44c2-866c-6dc6c21556ba admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.345033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.345678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.345964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.346273 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] PATH_INFO: `/v3/projects/6734231e940c4b12b8267eebd3d7a8c1/users/9aca2e99fa6d4d0d9535cd731b0e53e5/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.362179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6734231e940c4b12b8267eebd3d7a8c1, user_id=9aca2e99fa6d4d0d9535cd731b0e53e5, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.365076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.434133 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1308/2588] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:52 2022] PUT /identity/v3/projects/6734231e940c4b12b8267eebd3d7a8c1/users/9aca2e99fa6d4d0d9535cd731b0e53e5/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:52.440180 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.440463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.440736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.468781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] MFA Rules not processed for user `9aca2e99fa6d4d0d9535cd731b0e53e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:52.469995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:52.533490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1281/2589] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:52.549440 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.593824 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:52.610568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa796ba7-586b-4ad4-be43-c7cf208d7f11 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.615160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.615753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.616023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.616290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.616917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.619381 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.725962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1309/2590] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:52 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:52.730802 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.741357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d34a-b8ce-4837-b105-a1106b518e08 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.796983 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.797582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.797855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.798148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.798796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.800697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.860258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1282/2591] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:52 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:52.864390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.874985 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a4106-9b6a-435e-a3d1-d81a72b90dfa admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.879600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.880225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.880505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.880774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.881412 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.883018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.893563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1310/2592] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:52.899765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.908561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9968c-68b3-467d-802e-d9ad375adabe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.913037 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.913668 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.913964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.914260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] PATH_INFO: `/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907/users/3dc1d632c15d4a9cb2358376ff1c11c1/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.929646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4cbe640f6c474d80c6f9bf7dfd5907, user_id=3dc1d632c15d4a9cb2358376ff1c11c1, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.932352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:53.010630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1283/2593] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:52 2022] PUT /identity/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907/users/3dc1d632c15d4a9cb2358376ff1c11c1/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:53.014679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:53.025238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca97b30-547f-4839-b7eb-3a99eb98e205 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:53.083780 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:53.084729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:53.085012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:53.085285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:53.085964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:53.088150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cb5d1-e51d-4e0c-a377-8b34c4ebe773 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:53.098105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1311/2594] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:53.102701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:53.113551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f5374-c825-4f4b-860e-cef4cc275613 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:53.118194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:53.123111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:53.123111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:53.123111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] PATH_INFO: `/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907/users/3dc1d632c15d4a9cb2358376ff1c11c1/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:53.123111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4cbe640f6c474d80c6f9bf7dfd5907, user_id=3dc1d632c15d4a9cb2358376ff1c11c1, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:53.124183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:53.135425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1284/2595] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:53 2022] PUT /identity/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907/users/3dc1d632c15d4a9cb2358376ff1c11c1/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:53.146793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:53.151858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:53.151858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:53.175989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] MFA Rules not processed for user `3dc1d632c15d4a9cb2358376ff1c11c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:53.177215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:53.250265 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1312/2596] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:53.261947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:53.263720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:53.270323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:53.324601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b381e4-b228-4aae-a8ba-733eb5410ce8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:53.328345 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:53.329001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:53.329301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:53.329552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:53.340288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:53.341864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:53.351828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1285/2597] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:53.795272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:53.796747 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:53.843396 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:53.858766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1cf19-4b95-4d8d-9163-27e15bb42284 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:53.862425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:53.863034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:53.863312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:53.863581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:53.865350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:53.927930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:53.929342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a586e4-dd49-4cf4-9031-aa8968de558c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:53.939260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1313/2598] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:56:54.014709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:54.026182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cefdb8-93ed-4e72-8939-cb37b4ddeddd service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:54.030990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:54.031723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:54.031997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:54.032366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] PATH_INFO: `/v3/users/9aca2e99fa6d4d0d9535cd731b0e53e5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:54.033498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aca2e99fa6d4d0d9535cd731b0e53e5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:54.035454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:54.195779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aca2e99fa6d4d0d9535cd731b0e53e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:54.209543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aca2e99fa6d4d0d9535cd731b0e53e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:54.216913 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aca2e99fa6d4d0d9535cd731b0e53e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:54.290889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aca2e99fa6d4d0d9535cd731b0e53e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:54.291510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aca2e99fa6d4d0d9535cd731b0e53e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:54.298252 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1286/2599] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:54 2022] DELETE /identity/v3/users/9aca2e99fa6d4d0d9535cd731b0e53e5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:54.302933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:54.303304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:54.303579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:54.324337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:54.325551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:54.392206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1314/2600] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:54.402386 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:54.403856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:54.463923 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9a507-0470-4f53-a910-b9d6951a5d77 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:54.467655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:54.468314 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:54.468630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:54.468886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:54.477852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:54.479323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:54.488922 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1287/2601] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:56:55.007779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:55.010140 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:55.031407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8bc7b-f4e6-4d3a-bff2-6d80a67b1241 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:55.039520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:55.054261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:55.054261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:55.054261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] PATH_INFO: `/v3/projects/6734231e940c4b12b8267eebd3d7a8c1` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:55.054261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6734231e940c4b12b8267eebd3d7a8c1)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:55.054261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:55.178037 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6734231e940c4b12b8267eebd3d7a8c1', 'request_id': 'req-c2439041-3383-41b8-978f-e07358ccb5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.185490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6734231e940c4b12b8267eebd3d7a8c1', 'request_id': 'req-c2439041-3383-41b8-978f-e07358ccb5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.192399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6734231e940c4b12b8267eebd3d7a8c1', 'request_id': 'req-c2439041-3383-41b8-978f-e07358ccb5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.267956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] The token cache is being invalidate because project 6734231e940c4b12b8267eebd3d7a8c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:55.268254 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.268921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.270490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1315/2602] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:55 2022] DELETE /identity/v3/projects/6734231e940c4b12b8267eebd3d7a8c1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:55.274748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:55.285006 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b01ca2-044a-4e2d-891d-35946652f12e service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:55.305489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:55.345294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:55.345951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:55.346259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:55.346545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] PATH_INFO: `/v3/users/3dc1d632c15d4a9cb2358376ff1c11c1` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:55.347739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc1d632c15d4a9cb2358376ff1c11c1)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:55.349689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:55.527535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc1d632c15d4a9cb2358376ff1c11c1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.541285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc1d632c15d4a9cb2358376ff1c11c1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.548285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc1d632c15d4a9cb2358376ff1c11c1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.621356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc1d632c15d4a9cb2358376ff1c11c1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.622489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bcd9bdf9-190a-4c10-8fd4-5f0998dbcec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc1d632c15d4a9cb2358376ff1c11c1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:55.629786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1288/2603] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:55 2022] DELETE /identity/v3/users/3dc1d632c15d4a9cb2358376ff1c11c1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:56.142150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:56.143969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:56.162453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2439041-3383-41b8-978f-e07358ccb5a8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:56.174542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:56.227802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:56.228498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.228775 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.229045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] PATH_INFO: `/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.230210 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4cbe640f6c474d80c6f9bf7dfd5907)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:56.232153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:56.348417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4cbe640f6c474d80c6f9bf7dfd5907', 'request_id': 'req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:56.354494 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4cbe640f6c474d80c6f9bf7dfd5907', 'request_id': 'req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:56.360665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4cbe640f6c474d80c6f9bf7dfd5907', 'request_id': 'req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:56.485750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] The token cache is being invalidate because project 3a4cbe640f6c474d80c6f9bf7dfd5907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:56.489801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:56.489801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-415bc7a4-b6f9-40b8-a664-d12de0e63e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:56.490912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1316/2604] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:56 2022] DELETE /identity/v3/projects/3a4cbe640f6c474d80c6f9bf7dfd5907 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:56.495990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.496496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.496612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.519986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:56.521224 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:56.590569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1289/2605] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:56.600637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.600939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.601206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.624924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:56.626150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:56.634877 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1317/2606] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:56.639298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:56.656324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147a427-6b46-41f4-a9c9-49b52208926e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:56.656731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:56.657334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.657609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.657874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.658544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:56.658840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:56.660620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:56.673089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1290/2607] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:56.677598 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:56.688658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7475-054f-46a9-bd7f-8703e046eac4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:56.694371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:56.694981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.695305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.695579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.696841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:56.698764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:56.823726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1318/2608] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:56.827850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:56.837971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecbed8-6062-4ce9-9660-8607bb9eb8b0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:56.865145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:56.896367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:56.896977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:56.897286 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:56.897628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:56.898218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:56.900145 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:56.988325 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1291/2609] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:56 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:56.992582 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.004078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49b6ba-5cf4-42a7-8cc1-09f39e442d24 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.008821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.009631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.009940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.010230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.010883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.012486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.025793 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1319/2610] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:57.031421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.043279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d37be-80e5-427b-bce1-0fd0fd5cbae1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.053443 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.055478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.055478 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.062854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.062854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.062854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24be99-ccca-4482-83fb-b25eabd6f666 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.079902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.085666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1292/2611] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:56:57.091067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847f0-4333-4759-90ea-77ef49475ee5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.096121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.096784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.097092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.097467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] PATH_INFO: `/v3/projects/3b183c0c26a24a519c58fd78097d4b9b/users/b9a4c396400241e2b7fb6de9fcd97e7f/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.115326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b183c0c26a24a519c58fd78097d4b9b, user_id=b9a4c396400241e2b7fb6de9fcd97e7f, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.118899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.217986 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1320/2612] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:57 2022] PUT /identity/v3/projects/3b183c0c26a24a519c58fd78097d4b9b/users/b9a4c396400241e2b7fb6de9fcd97e7f/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:57.220952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.221440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.221872 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.268631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] MFA Rules not processed for user `b9a4c396400241e2b7fb6de9fcd97e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:57.269889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:57.350764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1293/2613] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:57.365607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.367417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:57.413200 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:57.428944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55a9af-6b7f-464f-8127-f24d92e144a9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.434307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.435540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.435540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.435540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.444107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.445522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.455243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1321/2614] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:56:57.478001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.551142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e4ef-95ee-47d8-a30e-4a3ea1769faf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.556561 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.557327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.557693 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.558033 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] PATH_INFO: `/v3/users/b9a4c396400241e2b7fb6de9fcd97e7f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.559289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a4c396400241e2b7fb6de9fcd97e7f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.561301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.779848 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:57.785543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a4c396400241e2b7fb6de9fcd97e7f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:57.798748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a4c396400241e2b7fb6de9fcd97e7f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:57.805548 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a4c396400241e2b7fb6de9fcd97e7f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:57.864169 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.865678 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:57.880499 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a58dd-d2ff-4860-824f-0578bed13619 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:57.885728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:57.886403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.886683 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.886958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] PATH_INFO: `/v3/users/e00238b83af948fe8752f998aa08b0b6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.888111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00238b83af948fe8752f998aa08b0b6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:57.890028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:57.931215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a4c396400241e2b7fb6de9fcd97e7f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:57.931865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18a302e9-da98-4590-8dd5-aad752695a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a4c396400241e2b7fb6de9fcd97e7f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:57.938159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1294/2615] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:57 2022] DELETE /identity/v3/users/b9a4c396400241e2b7fb6de9fcd97e7f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:57.942490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.942773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.945612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.962410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:57.963661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:58.023513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1295/2616] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:58.033717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.035155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:58.093433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e419b1-cbf6-4f82-92b0-a54d130b22ee None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.097722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.098428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.098778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.099116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.101527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00238b83af948fe8752f998aa08b0b6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.109199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.110756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.116542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00238b83af948fe8752f998aa08b0b6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.121332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1296/2617] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:56:58.124467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00238b83af948fe8752f998aa08b0b6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.125232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00238b83af948fe8752f998aa08b0b6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.216540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00238b83af948fe8752f998aa08b0b6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.223112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1322/2618] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:56:57 2022] DELETE /identity/v3/users/e00238b83af948fe8752f998aa08b0b6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:58.601186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.603250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:58.616680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939ecc0-e013-4877-b8b8-df0cedf5fb58 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.664101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.666034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:58.679207 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.683274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.683274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.683274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] PATH_INFO: `/v3/projects/0b33edf3f3594215a86fa6d2d0e68c77` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.683274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b33edf3f3594215a86fa6d2d0e68c77)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.684483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.686185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:58.738282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103bb07-a339-4a6e-8796-9ac3b8c7726c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.742857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.746308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.746308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.746308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] PATH_INFO: `/v3/projects/3b183c0c26a24a519c58fd78097d4b9b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.746308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b183c0c26a24a519c58fd78097d4b9b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.747021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.833457 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b33edf3f3594215a86fa6d2d0e68c77', 'request_id': 'req-a02abd72-3b29-4117-9915-6a13ae7dba5d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.868915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b183c0c26a24a519c58fd78097d4b9b', 'request_id': 'req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.875306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b183c0c26a24a519c58fd78097d4b9b', 'request_id': 'req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.876492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:58.881309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b183c0c26a24a519c58fd78097d4b9b', 'request_id': 'req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.970788 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b33edf3f3594215a86fa6d2d0e68c77', 'request_id': 'req-a02abd72-3b29-4117-9915-6a13ae7dba5d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.977812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b33edf3f3594215a86fa6d2d0e68c77', 'request_id': 'req-a02abd72-3b29-4117-9915-6a13ae7dba5d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.989851 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] The token cache is being invalidate because project 0b33edf3f3594215a86fa6d2d0e68c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:58.990173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.990822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a02abd72-3b29-4117-9915-6a13ae7dba5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:58.992302 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1297/2619] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:58 2022] DELETE /identity/v3/projects/0b33edf3f3594215a86fa6d2d0e68c77 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:58.997656 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d6d8-f3a2-4509-8e51-b05858ec1596 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.997942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d6d8-f3a2-4509-8e51-b05858ec1596 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.998262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524d6d8-f3a2-4509-8e51-b05858ec1596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.017710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c524d6d8-f3a2-4509-8e51-b05858ec1596 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.018963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c524d6d8-f3a2-4509-8e51-b05858ec1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.056700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] The token cache is being invalidate because project 3b183c0c26a24a519c58fd78097d4b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:56:59.057023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:59.057688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-339cc635-43c8-4d4c-aa34-62ac1b3a6c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:56:59.059197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1323/2620] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:56:58 2022] DELETE /identity/v3/projects/3b183c0c26a24a519c58fd78097d4b9b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:59.063947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da293-81c4-4e51-baeb-20469cf3f8f9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.064226 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da293-81c4-4e51-baeb-20469cf3f8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.064519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8da293-81c4-4e51-baeb-20469cf3f8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.077633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1298/2621] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:59.083595 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9f8da293-81c4-4e51-baeb-20469cf3f8f9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.085311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9f8da293-81c4-4e51-baeb-20469cf3f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.085852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.086379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.087535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.093535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1324/2622] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:59.103439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.103726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.103995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.108242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.109462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.117632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1299/2623] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:59.122651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.124236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.126408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.131716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4e7d5-c09c-43d0-a05e-d9413ce83a09 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.135496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1325/2624] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:56:59.140410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.140410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.140410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.140410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.140957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:59.140957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.140957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.143431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.151976 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aee72-48ca-45f6-b71b-ad8cac1c2610 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.152578 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1300/2625] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:56:59.156808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.157072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.157744 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.158122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.158474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.159217 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:59.159820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.161960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.167271 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c866f6-854d-436e-8a58-4e17bc3b3717 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.173239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.173824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.174133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.174412 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.175040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.175466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1326/2626] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:56:59.177048 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.180027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.190721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd7518-b830-4cbb-bdd3-bf2d96890d23 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.195616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.196243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.196522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.196792 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.197425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.199321 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.298040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1301/2627] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:59.302186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.313441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1327/2628] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:59.314600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.352241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:59.368191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5fc18-45cf-46eb-8b8e-85441188655e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.384957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee60ff-f2cc-49a0-8aa8-c51df04e5faa admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.389888 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.390526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.390808 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.391083 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.391724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.393625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.426754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.427369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.427652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.427930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.428549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.430499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.548471 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1328/2629] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:59.552565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.562145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28a93-8cd5-40ce-8925-3c7efef58cb5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.566821 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.567392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.567671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.567943 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.568575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.570279 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.578092 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1302/2630] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:56:59.582170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.586118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1329/2631] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:59.587618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.593430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2583f5e-8b73-4246-b577-3e8805cfb453 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.597540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2a430-235f-42d0-b5d0-f8b955f46505 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.598702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.605726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.605726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.605726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.605726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.605726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.606609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.606609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.606609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.606609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.606990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.606990 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.616698 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1303/2632] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:59.616866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.623300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1330/2633] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:59.624074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.629619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fdd30-3659-4ec1-8c83-87465f8f7070 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.643667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3d88c-18c8-42c4-adcc-10ed568708d7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.643850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.643850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.643850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.643850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.644256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.644256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.644444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.644444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.644444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.644444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] PATH_INFO: `/v3/projects/55d7e7f08a4f4c848204193b02f751c0/users/de0689b1c8df42f6a818224f4a0321df/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.659617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d7e7f08a4f4c848204193b02f751c0, user_id=de0689b1c8df42f6a818224f4a0321df, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.662320 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d8f7a-85fe-4feb-8d32-49f18a150c2f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.776858 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1304/2634] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:56:59.781191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.788367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1331/2635] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:59 2022] PUT /identity/v3/projects/55d7e7f08a4f4c848204193b02f751c0/users/de0689b1c8df42f6a818224f4a0321df/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:59.793777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aa3cb-4f25-4c03-b4ef-1b16771da79e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:59.807359 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.807638 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.807907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.836506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] MFA Rules not processed for user `de0689b1c8df42f6a818224f4a0321df`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.840236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.850832 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:59.851492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.851770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.852042 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] PATH_INFO: `/v3/projects/587c9bca942044cf8e127df11e3c37a2/users/f6e5f660250643e6a10312b5a74d7e09/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.866288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587c9bca942044cf8e127df11e3c37a2, user_id=f6e5f660250643e6a10312b5a74d7e09, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:59.867668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:59.869030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244950d7-566d-4c74-b49a-f874f01c7419 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:59.908922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1332/2636] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:56:59.921557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:59.922932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.932080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1305/2637] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:56:59 2022] PUT /identity/v3/projects/587c9bca942044cf8e127df11e3c37a2/users/f6e5f660250643e6a10312b5a74d7e09/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:56:59.943444 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:59.943724 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:59.943991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:59.971099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] MFA Rules not processed for user `f6e5f660250643e6a10312b5a74d7e09`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:59.972216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:59.982193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a924aa-6450-4fec-94eb-eb33a60af765 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:00.006901 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:00.036558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66182402-de05-4cfe-a560-1946d403596d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d651db5bf74166bff093d317d83a6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:00.037162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66182402-de05-4cfe-a560-1946d403596d service glance] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.037441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66182402-de05-4cfe-a560-1946d403596d service glance] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.037711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66182402-de05-4cfe-a560-1946d403596d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.038346 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1306/2638] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:00.050250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:00.051776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:00.101542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66182402-de05-4cfe-a560-1946d403596d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:00.103500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66182402-de05-4cfe-a560-1946d403596d service glance] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:00.112481 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1333/2639] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:56:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:57:00.129871 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892d11d-c349-4365-ad48-b0019da5b3db None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:00.136332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:00.136838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.138411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.138411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.146597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:00.148586 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae700f57-6740-4466-92d5-566229d25c80 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:00.157389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1307/2640] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:00.388427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66182402-de05-4cfe-a560-1946d403596d service glance] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:00.429343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:00.450574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66182402-de05-4cfe-a560-1946d403596d service glance] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:00.455163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:00.455788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.456067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.456339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] PATH_INFO: `/v3/users/f6e5f660250643e6a10312b5a74d7e09` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.457461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e5f660250643e6a10312b5a74d7e09)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:00.459635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:00.699771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e5f660250643e6a10312b5a74d7e09'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:00.712563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e5f660250643e6a10312b5a74d7e09'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:00.719513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e5f660250643e6a10312b5a74d7e09'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:00.720192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e5f660250643e6a10312b5a74d7e09'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:00.799236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e5f660250643e6a10312b5a74d7e09'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:00.805513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1334/2641] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:57:00 2022] DELETE /identity/v3/users/f6e5f660250643e6a10312b5a74d7e09 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:00.810398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.810879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.811008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.830456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:00.831678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:00.890278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1308/2642] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:00.900875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:00.902300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:00.959945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bd38d-9d54-4174-a5ab-b6d47dc84982 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:00.963444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:00.964093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.964367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.964640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.973645 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:00.975093 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a662f-c462-404d-a8d5-cbb2d322971e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:00.984568 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1335/2643] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:57:01.419933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.420255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:01.432964 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fab53-9a31-4d57-baa6-694de03009a6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.437605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.438269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.438547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.438816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] PATH_INFO: `/v3/projects/587c9bca942044cf8e127df11e3c37a2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.439930 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=587c9bca942044cf8e127df11e3c37a2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.441948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.606910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587c9bca942044cf8e127df11e3c37a2', 'request_id': 'req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:01.702152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587c9bca942044cf8e127df11e3c37a2', 'request_id': 'req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:01.703634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:01.708822 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587c9bca942044cf8e127df11e3c37a2', 'request_id': 'req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:01.721101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] The token cache is being invalidate because project 587c9bca942044cf8e127df11e3c37a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:57:01.721391 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:01.722087 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-05f31aa9-1f75-4598-b3f5-3244aadb8a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:01.723558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1309/2644] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:57:01 2022] DELETE /identity/v3/projects/587c9bca942044cf8e127df11e3c37a2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:01.729118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.729614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.729722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.749420 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:01.750653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:01.807742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:01.816928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1336/2645] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:57:01.824982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.826454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.826454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.846804 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:01.848173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:01.857900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1310/2646] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:01.862116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.871914 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220037cf-4fdf-467a-94ec-7a7be68f60e4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.877229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.877840 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.878150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.878424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.879085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:57:01.879411 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.883822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.893064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1337/2647] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:57:01.897581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.908119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6954701-5fbc-4cc1-bf6c-f3b541294423 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.912841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.913428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.913707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.913975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.914731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.917361 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.994543 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1311/2648] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:57:01 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:01.998609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.008899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb652-1407-4f5d-81d8-65829ae3f85e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.066870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.067477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.067753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.068020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.068754 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.070692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.185732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1338/2649] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:01 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:02.187138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.198400 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e89ac-a75b-42f8-98c4-b7c34309e89f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.203060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.203697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.203977 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.204247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.204891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.206516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.217844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1312/2650] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:02.221873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.233676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37c99a-3698-4a9e-a12e-fbe6ffe6da26 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.238418 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.239074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.239349 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.239622 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.240268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.241846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e8c7f-081f-42b1-92bc-b74b07df08df admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.252648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1339/2651] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:02.256831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.268273 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524cfc-a9fd-456d-be09-f859f4f337ba admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.272971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.273630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.273910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.274254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] PATH_INFO: `/v3/projects/da936287f1bf452891dad5254f1e5164/users/06d2f7515108418593bcec9549c40b02/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.290462 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da936287f1bf452891dad5254f1e5164, user_id=06d2f7515108418593bcec9549c40b02, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.293131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.376522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1313/2652] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:02 2022] PUT /identity/v3/projects/da936287f1bf452891dad5254f1e5164/users/06d2f7515108418593bcec9549c40b02/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:02.387646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.387900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.388150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.421644 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] MFA Rules not processed for user `06d2f7515108418593bcec9549c40b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:02.422895 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:02.489830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1340/2653] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:02.494885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.557068 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa2729-c59e-42ae-ac7e-f41280adb78c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.561686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.562311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.562590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.562859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.563481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.565373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.692784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1314/2654] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:57:02 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:02.697616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.708777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972376e-81ee-40b9-aad0-cf35d34793ec None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.758585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:02.765926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.766567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.766848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.767194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.767767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.769767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.880201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1341/2655] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:02 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:57:02.884130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.894986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c900c7-be95-4fc6-b9c5-0f252540c7fa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.900440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.900980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.901260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.901532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.902284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.903950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:02.915643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1315/2656] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:02.919898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:02.930082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d432ae-88d0-4e11-afb0-c472fa55c46c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:02.935184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:02.935826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:02.936105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:02.943525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] PATH_INFO: `/v3/projects/66c7330d355042848b3bc77af7930697/users/d4facdb3178a42adace33e92062e0283/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:02.952885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c7330d355042848b3bc77af7930697, user_id=d4facdb3178a42adace33e92062e0283, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:02.955559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:03.023023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1342/2657] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:02 2022] PUT /identity/v3/projects/66c7330d355042848b3bc77af7930697/users/d4facdb3178a42adace33e92062e0283/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:57:03.027362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:03.037562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc0d91-08b1-4308-9196-02956afcbbfd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:03.088786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:03.095588 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:03.102101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:03.102101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:03.102101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:03.102101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:03.102101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee09556-9c03-4cf9-9a8f-9f0cd3527718 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:03.108833 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1316/2658] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:57:03.113407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:03.125648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473f45-92fc-4b5a-b3a6-38ff2d803ee5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:03.130637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:03.130955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:03.131148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:03.131415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] PATH_INFO: `/v3/projects/66c7330d355042848b3bc77af7930697/users/d4facdb3178a42adace33e92062e0283/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:03.133335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c7330d355042848b3bc77af7930697, user_id=d4facdb3178a42adace33e92062e0283, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:03.135995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:03.146543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1343/2659] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:03 2022] PUT /identity/v3/projects/66c7330d355042848b3bc77af7930697/users/d4facdb3178a42adace33e92062e0283/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:03.158311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:03.158852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:03.159321 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:03.188568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] MFA Rules not processed for user `d4facdb3178a42adace33e92062e0283`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:03.189810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:03.264918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1317/2660] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:03.276803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:03.278528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:03.335830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be6e9-4070-4f4d-9753-219cff45b464 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:03.339975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:03.340577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:03.340854 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:03.341126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:03.349947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:03.351377 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:03.362570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1344/2661] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:03.542909 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:03.544306 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:03.556111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec1c67-554c-4ea9-8ea0-7494fa86edf6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:03.559739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:03.560323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:03.560595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:03.560862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:03.619243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:03.620680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:03.630587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1318/2662] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:04.388553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:04.397520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15940e48-20b8-4683-ba44-f5096573ab0a service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:04.401180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:04.401773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:04.402062 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:04.402369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:04.404178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:04.466786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:04.468107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fceb6ac-58c2-4b7e-a8a3-1acb216fec3c service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:04.478124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1345/2663] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:08.620027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:08.682702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3f171-d716-44a6-b382-dc025a639efa service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:08.687077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:08.687697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:08.687973 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:08.688242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] PATH_INFO: `/v3/users/06d2f7515108418593bcec9549c40b02` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:08.689344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d2f7515108418593bcec9549c40b02)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:08.691280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:08.707293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:08.895001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d2f7515108418593bcec9549c40b02'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:08.907921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d2f7515108418593bcec9549c40b02'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:08.914826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d2f7515108418593bcec9549c40b02'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:08.965539 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d2f7515108418593bcec9549c40b02'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:08.966141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d2f7515108418593bcec9549c40b02'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:08.972470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1319/2664] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:57:08 2022] DELETE /identity/v3/users/06d2f7515108418593bcec9549c40b02 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:08.977034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:08.977527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:08.977641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:08.997520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:08.998772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:09.046252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:09.062276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1346/2665] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:09.071989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:09.074488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:09.132582 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64152-f41e-4fc7-abe9-93bc4708713b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:09.136890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:09.137502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:09.137778 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:09.138047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:09.147521 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:09.148944 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:09.158373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1320/2666] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:57:09.572619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:09.576477 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:09.588163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207033b5-115e-441a-80aa-c7fcd24143c8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:09.592919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:09.593553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:09.593834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:09.594132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] PATH_INFO: `/v3/projects/da936287f1bf452891dad5254f1e5164` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:09.595258 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da936287f1bf452891dad5254f1e5164)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:09.597168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:09.709159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da936287f1bf452891dad5254f1e5164', 'request_id': 'req-cb151d4d-8bcc-4979-8841-17aa71f44594'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:09.715928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da936287f1bf452891dad5254f1e5164', 'request_id': 'req-cb151d4d-8bcc-4979-8841-17aa71f44594'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:09.721675 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da936287f1bf452891dad5254f1e5164', 'request_id': 'req-cb151d4d-8bcc-4979-8841-17aa71f44594'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:09.799428 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] The token cache is being invalidate because project da936287f1bf452891dad5254f1e5164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:57:09.799625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:09.800274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:09.801748 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1347/2667] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:57:09 2022] DELETE /identity/v3/projects/da936287f1bf452891dad5254f1e5164 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:09.806031 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:09.817279 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead6732-24de-489e-ba88-222ae57d770c service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:09.874072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:09.874737 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:09.875016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:09.875289 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] PATH_INFO: `/v3/users/d4facdb3178a42adace33e92062e0283` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:09.876442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4facdb3178a42adace33e92062e0283)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:09.878425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:10.014171 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:10.020762 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4facdb3178a42adace33e92062e0283'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.032579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4facdb3178a42adace33e92062e0283'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.039984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4facdb3178a42adace33e92062e0283'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.108324 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4facdb3178a42adace33e92062e0283'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.108950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7606da28-0e50-40b6-8a61-f9e0531cf626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4facdb3178a42adace33e92062e0283'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.115284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1321/2668] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:57:09 2022] DELETE /identity/v3/users/d4facdb3178a42adace33e92062e0283 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:10.537958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:10.542537 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:10.554785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d4d-8bcc-4979-8841-17aa71f44594 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:10.625557 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:10.626235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:10.626535 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:10.626875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] PATH_INFO: `/v3/projects/66c7330d355042848b3bc77af7930697` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:10.627981 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c7330d355042848b3bc77af7930697)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:10.629858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:10.753646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c7330d355042848b3bc77af7930697', 'request_id': 'req-c3d285d5-e295-4df6-a7c4-128076a178e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.759921 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c7330d355042848b3bc77af7930697', 'request_id': 'req-c3d285d5-e295-4df6-a7c4-128076a178e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.765958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c7330d355042848b3bc77af7930697', 'request_id': 'req-c3d285d5-e295-4df6-a7c4-128076a178e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.828318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] The token cache is being invalidate because project 66c7330d355042848b3bc77af7930697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:57:10.828642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.829256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c3d285d5-e295-4df6-a7c4-128076a178e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:10.830795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1348/2669] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:57:10 2022] DELETE /identity/v3/projects/66c7330d355042848b3bc77af7930697 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:10.836084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:10.836535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:10.836633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:10.857283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:10.862849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:10.918038 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1322/2670] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:10.926014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:10.926778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:10.926778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:10.934474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:10.947135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:10.948855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:10.956950 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1349/2671] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:10.961598 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:10.971937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c90a5-4882-47b1-89d9-7bb9e8a5852b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:10.976651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:10.977270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:10.977547 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:10.977816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:10.978505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:57:10.978827 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:10.980507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:10.993235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1323/2672] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:57:10.997588 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.007119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c7cd-f95c-4778-b9f8-66af31b821e1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.012445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.013051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.013326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.013594 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.014270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.016157 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.116153 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1350/2673] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:57:10 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:57:11.124298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.134894 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892a9fc-b3f5-4999-ac36-1da01d1e046c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.189969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.190593 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.190870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.191140 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.191765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.193652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.269304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1324/2674] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:11 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:11.273463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.283988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe36588-3f5d-4db5-9b4e-e62b79404bed admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.288575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.289192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.289468 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.289786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.290461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.292016 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.301722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1351/2675] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:11.305605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.316066 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf62a-84f5-41c2-bb84-3daa8da57e76 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.320754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.321336 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.321618 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.321890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.322574 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.324826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da590aa-8212-4be1-8e51-d089b2d34511 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.335769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1325/2676] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:11.339844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.349597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319eb83-6e2e-479f-ae53-cf10ad489fb1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.354187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.354826 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.355106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.355384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] PATH_INFO: `/v3/projects/5bf42bbb9f094415ad2f12054b28d271/users/d8898c2cd1994e5b8f5f611f22db390a/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.371106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf42bbb9f094415ad2f12054b28d271, user_id=d8898c2cd1994e5b8f5f611f22db390a, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.374502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.468820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1352/2677] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:11 2022] PUT /identity/v3/projects/5bf42bbb9f094415ad2f12054b28d271/users/d8898c2cd1994e5b8f5f611f22db390a/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:11.486996 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.487278 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.487565 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.515027 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] MFA Rules not processed for user `d8898c2cd1994e5b8f5f611f22db390a`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:11.516234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:11.525436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:11.582535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1326/2678] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:11.587561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.655646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063227d3-b019-4e5d-86f1-69b4b5c2c3e0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.660457 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.661064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.661369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.661788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.662516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.664668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.790735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1353/2679] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:57:11 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:11.794836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:11.805669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cd38d-9f08-46b5-b35e-e732c91bfb44 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:11.863610 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:11.864408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:11.864527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:11.864735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:11.865377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:11.867403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:11.989287 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1327/2680] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:11 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:11.993235 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:12.005023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e9f3b-3fd7-4f3f-b89a-0a4b26fee5e0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:12.009652 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:12.010223 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.010500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.010769 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.011413 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:12.013626 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:12.024499 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1354/2681] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:12.030152 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:12.039026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973f6b3-562c-4a4b-9d8f-190ba787d6a8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:12.043659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:12.044284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.044556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.044828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] PATH_INFO: `/v3/projects/1ae39790d7f449598ca9fbe267557399/users/0f92a2ba61da4519ae1e66059b01ca33/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.067001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae39790d7f449598ca9fbe267557399, user_id=0f92a2ba61da4519ae1e66059b01ca33, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:12.070015 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:12.177609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1328/2682] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:12 2022] PUT /identity/v3/projects/1ae39790d7f449598ca9fbe267557399/users/0f92a2ba61da4519ae1e66059b01ca33/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:57:12.181815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:12.191774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8a7a3-f526-4342-8754-e0b07554e0cb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:12.249142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:12.249666 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.249949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.250249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.250883 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:12.252461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bbccf-28d6-4078-994e-935dc68ec0ea admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:12.262454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1355/2683] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:12.267322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:12.279080 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6249804-b346-4b2f-8bd4-4191a091a10b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:12.283717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:12.284358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.284638 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.284908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] PATH_INFO: `/v3/projects/1ae39790d7f449598ca9fbe267557399/users/0f92a2ba61da4519ae1e66059b01ca33/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.286842 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae39790d7f449598ca9fbe267557399, user_id=0f92a2ba61da4519ae1e66059b01ca33, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:12.289434 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:12.300382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1329/2684] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:12 2022] PUT /identity/v3/projects/1ae39790d7f449598ca9fbe267557399/users/0f92a2ba61da4519ae1e66059b01ca33/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:12.311729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.312025 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.312298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.332618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:12.341926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] MFA Rules not processed for user `0f92a2ba61da4519ae1e66059b01ca33`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:12.343145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:12.413269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1356/2685] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:12.424175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:12.425888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:12.485771 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc2eaa-2e17-4105-b495-3103ed5a248a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:12.489385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:12.489988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:12.490298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:12.490601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:12.500182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:12.501549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d340-bf5e-4863-869c-60af6ff22b3b service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:12.511005 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1330/2686] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:13.707445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:13.799102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912119d-48bf-47dd-a9fa-9143c32d7c7a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:13.804495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:13.805278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:13.805796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:13.805796 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] PATH_INFO: `/v3/users/d8898c2cd1994e5b8f5f611f22db390a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:13.806982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8898c2cd1994e5b8f5f611f22db390a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:13.808906 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:14.037290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8898c2cd1994e5b8f5f611f22db390a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.050041 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8898c2cd1994e5b8f5f611f22db390a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.057256 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8898c2cd1994e5b8f5f611f22db390a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.058079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8898c2cd1994e5b8f5f611f22db390a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.154573 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8898c2cd1994e5b8f5f611f22db390a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.160988 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1357/2687] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:57:13 2022] DELETE /identity/v3/users/d8898c2cd1994e5b8f5f611f22db390a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:14.165399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:14.165954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:14.165954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:14.178905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:14.185267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:14.186503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:14.245116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1331/2688] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:14.255719 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:14.257122 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:14.272354 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:14.315714 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b72fa5-17be-4260-8b35-af53bf2ba9fc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:14.319331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:14.319933 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:14.320212 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:14.320486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:14.329970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:14.331406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:14.341290 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1358/2689] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:14.784241 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:14.785717 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:14.799702 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d7109-24ba-45ab-bf4f-3b5ffabc9ec8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:14.808294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:14.808952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:14.809214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:14.810078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] PATH_INFO: `/v3/projects/5bf42bbb9f094415ad2f12054b28d271` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:14.811215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf42bbb9f094415ad2f12054b28d271)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:14.813165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:14.921262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf42bbb9f094415ad2f12054b28d271', 'request_id': 'req-85c48c7b-0d44-4231-b718-32ffd3163872'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.976617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf42bbb9f094415ad2f12054b28d271', 'request_id': 'req-85c48c7b-0d44-4231-b718-32ffd3163872'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.983124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf42bbb9f094415ad2f12054b28d271', 'request_id': 'req-85c48c7b-0d44-4231-b718-32ffd3163872'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.994604 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] The token cache is being invalidate because project 5bf42bbb9f094415ad2f12054b28d271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:57:14.994900 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.995587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:14.997256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1332/2690] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:57:14 2022] DELETE /identity/v3/projects/5bf42bbb9f094415ad2f12054b28d271 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:15.001300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:15.010912 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723e2ee-4734-4089-ba6d-32ccf1dd1623 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:15.067437 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:15.068061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:15.068335 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:15.068603 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] PATH_INFO: `/v3/users/0f92a2ba61da4519ae1e66059b01ca33` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:15.069710 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f92a2ba61da4519ae1e66059b01ca33)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:15.071625 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:15.219278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f92a2ba61da4519ae1e66059b01ca33'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.231319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f92a2ba61da4519ae1e66059b01ca33'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.238075 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f92a2ba61da4519ae1e66059b01ca33'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.238757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f92a2ba61da4519ae1e66059b01ca33'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.279964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-57b48d2c-f727-42f6-9d6e-07d4b5f6d4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f92a2ba61da4519ae1e66059b01ca33'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.285849 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1359/2691] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:57:15 2022] DELETE /identity/v3/users/0f92a2ba61da4519ae1e66059b01ca33 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:15.675284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:15.676748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:15.690679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48c7b-0d44-4231-b718-32ffd3163872 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:15.750043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:15.750792 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:15.751067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:15.751332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] PATH_INFO: `/v3/projects/1ae39790d7f449598ca9fbe267557399` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:15.752458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae39790d7f449598ca9fbe267557399)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:15.754437 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:15.829761 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:15.841703 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae39790d7f449598ca9fbe267557399', 'request_id': 'req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.908863 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae39790d7f449598ca9fbe267557399', 'request_id': 'req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.916064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae39790d7f449598ca9fbe267557399', 'request_id': 'req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.929167 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] The token cache is being invalidate because project 1ae39790d7f449598ca9fbe267557399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:57:15.929454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.930117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ada7d84a-9ae8-421d-bed6-ebaa7ea69d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:57:15.931645 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1333/2692] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:57:15 2022] DELETE /identity/v3/projects/1ae39790d7f449598ca9fbe267557399 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:15.937237 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:15.937520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:15.937818 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:15.957718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:15.958971 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:15.988963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:16.017636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1360/2693] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:16.025058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.025366 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.025680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.046674 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:16.047860 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:16.055835 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1334/2694] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:16.060624 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.069994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0517c-1e9b-4cbe-bbe0-8fbe325b1471 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.075342 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.075946 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.076225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.076500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.077154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:57:16.077461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.079088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.092357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1361/2695] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:57:16.095474 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.104875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f38-8e6d-4284-be4a-a9533d3a2035 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.110295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.111142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.111425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.111694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.112440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.114353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.344052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1335/2696] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:57:16 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:16.348297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.358096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0b412-6e27-4433-9a9d-8867e769f808 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.415352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.415966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.416240 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.416513 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.417146 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.419116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.514902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1362/2697] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:16 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:16.519234 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.529679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16426ae-b6fe-4fa3-a342-1a75d038d7d2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.534440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.535024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.535301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.535649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.536296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.537884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.547834 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1336/2698] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:57:16.552047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.562550 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8fbee-9321-4d6e-b667-bd9746135e4f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.567238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.567828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.568106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.568380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.569053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.570696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d84f77-0064-49e3-a23d-3b9250fe1ade admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.581873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1363/2699] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:16.586422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.596536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461887cc-d5ae-40c1-b14a-d15100adc4d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.601238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.602064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.602388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.602658 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] PATH_INFO: `/v3/projects/ea38fd21fcaf4f468cef5cc20f10f6e3/users/233ac9ac09fd49d2b44c0d83baaad9bb/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.617572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea38fd21fcaf4f468cef5cc20f10f6e3, user_id=233ac9ac09fd49d2b44c0d83baaad9bb, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.620205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.714926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1337/2700] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:57:16 2022] PUT /identity/v3/projects/ea38fd21fcaf4f468cef5cc20f10f6e3/users/233ac9ac09fd49d2b44c0d83baaad9bb/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:57:16.727731 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b748d2-60cb-4c08-a70f-56cc1153578d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.728126 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b748d2-60cb-4c08-a70f-56cc1153578d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.728498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b748d2-60cb-4c08-a70f-56cc1153578d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.756701 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-77b748d2-60cb-4c08-a70f-56cc1153578d None None] MFA Rules not processed for user `233ac9ac09fd49d2b44c0d83baaad9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:16.757953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-77b748d2-60cb-4c08-a70f-56cc1153578d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:16.820771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1364/2701] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:16.832410 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:16.834131 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:16.890721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48c2a-97d1-4d07-b83e-a234c3d5df02 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:16.894344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:16.895376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:16.895376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:16.895580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:16.904404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:16.905908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:16.914767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1338/2702] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:57:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:57:20.198433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:20.202474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:20.202474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:20.218700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] MFA Rules not processed for user `1c081d9bf3744177a2d2d50962a38027`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:20.219907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:20.244805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:20.273862 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1365/2703] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:57:26.910883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:30.314082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=11 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:03.794409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:03.798984 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:03.889350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbcc10-3dfc-4ae7-83c7-fe0e4afb7419 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:03.895021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:03.895633 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:03.895914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:03.896186 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:03.898127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:03.912735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:03.914435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e358a3-685e-416e-a8a7-8ee337473531 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:03.926817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1339/2704] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:05.234179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:05.297236 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022936d-99aa-4b1e-a207-c9b2d9f6de12 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:05.301859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:05.302514 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:05.302795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:05.303072 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] PATH_INFO: `/v3/users/233ac9ac09fd49d2b44c0d83baaad9bb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:05.304183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=233ac9ac09fd49d2b44c0d83baaad9bb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:05.306103 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:05.511314 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233ac9ac09fd49d2b44c0d83baaad9bb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:05.523172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233ac9ac09fd49d2b44c0d83baaad9bb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:05.530830 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233ac9ac09fd49d2b44c0d83baaad9bb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:05.534517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233ac9ac09fd49d2b44c0d83baaad9bb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:05.605682 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233ac9ac09fd49d2b44c0d83baaad9bb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:05.611864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1366/2705] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:05 2022] DELETE /identity/v3/users/233ac9ac09fd49d2b44c0d83baaad9bb => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:05.620630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:05.620630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:05.620630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:05.638883 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:05.640592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:05.704016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1340/2706] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:05.714182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:05.716096 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:05.796903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592f4f2-b05e-41ab-88e2-1e2802534359 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:05.796903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:05.796903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:05.796903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:05.797343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:05.799975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:05.801394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64b335-81fb-4763-8063-84580591f095 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:05.810807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1367/2707] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:06.248845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:06.255616 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:06.263731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e5a6-8d5e-4133-9a0e-eb913cc08bab None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:06.269969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:06.270643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.270935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.271284 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] PATH_INFO: `/v3/projects/ea38fd21fcaf4f468cef5cc20f10f6e3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.272358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea38fd21fcaf4f468cef5cc20f10f6e3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:06.275557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:06.401960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea38fd21fcaf4f468cef5cc20f10f6e3', 'request_id': 'req-47ab0734-550c-4b8d-9057-078fd4e35bb9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:06.520752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea38fd21fcaf4f468cef5cc20f10f6e3', 'request_id': 'req-47ab0734-550c-4b8d-9057-078fd4e35bb9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:06.527694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea38fd21fcaf4f468cef5cc20f10f6e3', 'request_id': 'req-47ab0734-550c-4b8d-9057-078fd4e35bb9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:06.539274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] The token cache is being invalidate because project ea38fd21fcaf4f468cef5cc20f10f6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:06.539562 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:06.540205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-47ab0734-550c-4b8d-9057-078fd4e35bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:06.541838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1341/2708] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:06 2022] DELETE /identity/v3/projects/ea38fd21fcaf4f468cef5cc20f10f6e3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:06.547156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.547889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.547889 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.568521 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:06.570580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:06.589119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:06.632653 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1368/2709] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:06.641577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.642235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.642235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.662083 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:06.663376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:06.672210 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1342/2710] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:06.676659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:06.686936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9a784-0ce1-44c7-871f-9e046b1e908e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:06.691648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:06.692247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.692529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.692803 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.693454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:06.693758 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:06.695374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:06.706732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1369/2711] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:06.711118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:06.716165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:06.721281 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748aee1f-6186-4800-9964-dac4abd8bab9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:06.726002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:06.726763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.727081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.727545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.728153 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:06.730601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:06.848237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1343/2712] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:06.852470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:06.863329 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d145b7f-366a-4a18-92a1-3c2199071518 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:06.921865 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:06.922512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.922794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.923189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.924047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:06.925893 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:06.996725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:06.998465 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.011436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56a40f-41cd-476f-8dea-73d0ce246595 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.016383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.017051 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.017320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.017589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] PATH_INFO: `/v3/users/de0689b1c8df42f6a818224f4a0321df` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.018739 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0689b1c8df42f6a818224f4a0321df)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.020634 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.058938 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1370/2713] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:07.062725 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.072544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b59407-7946-402e-ac90-9623d067cb12 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.077231 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.077844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.078155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.078434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.079117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.080807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.094081 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1371/2714] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:07.097915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.108583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968245ee-5ad8-4dc8-8487-430ec5420e07 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.114042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.114685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.114945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.115336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.115980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.117576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.130942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1372/2715] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:07.131977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.141801 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a019d8-da06-4654-956a-a368b9d7aedc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.146565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.147208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.147488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.147764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] PATH_INFO: `/v3/projects/6ce35766a30e43d0a87e281fd0fdec81/users/6a332258680f4c7ea105432cad23deb6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.163474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce35766a30e43d0a87e281fd0fdec81, user_id=6a332258680f4c7ea105432cad23deb6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.167502 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.267976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0689b1c8df42f6a818224f4a0321df'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:07.269766 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1373/2716] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:07 2022] PUT /identity/v3/projects/6ce35766a30e43d0a87e281fd0fdec81/users/6a332258680f4c7ea105432cad23deb6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:07.270623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.281728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.282251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0689b1c8df42f6a818224f4a0321df'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:07.291071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0689b1c8df42f6a818224f4a0321df'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:07.340546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530a68a-c7df-4dce-9803-64660e828230 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.344127 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.344722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.344995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.345260 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.346966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.352858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:07.383949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0689b1c8df42f6a818224f4a0321df'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:07.388467 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-67f4e9fe-a375-4575-a69e-642d6794b2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0689b1c8df42f6a818224f4a0321df'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:07.391718 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1344/2717] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:06 2022] DELETE /identity/v3/users/de0689b1c8df42f6a818224f4a0321df => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:07.393029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.393390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.393733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.410996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.412512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d759a7a-e676-488e-9d9b-93316a72bf92 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.422546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] MFA Rules not processed for user `6a332258680f4c7ea105432cad23deb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:07.422760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1374/2718] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:07.423935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.424090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.424406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.424807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.444218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:07.445617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.502421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1345/2719] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:07.510648 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1375/2720] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:07.513617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.515246 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.519954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.521699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.559733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:07.571911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65de95-a8d4-45b7-beea-3edff7a9d5aa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.576196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.576865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.577148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.577417 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.580970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35905cb0-3d08-41d5-9540-e7c63bc73264 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.589326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.589326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.589326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.589326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.589781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.589781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.597641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.599233 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd73543-16dc-457a-bf1c-8af626c3b3ab service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.606347 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1346/2721] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:07.609939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1376/2722] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:08.191918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:08.193889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:08.210639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24622-c111-43d6-ba21-723c1a9529d7 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:08.217972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:08.218643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.219061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.219215 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] PATH_INFO: `/v3/projects/55d7e7f08a4f4c848204193b02f751c0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.220970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d7e7f08a4f4c848204193b02f751c0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:08.222327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:08.363337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d7e7f08a4f4c848204193b02f751c0', 'request_id': 'req-98146ac1-d17d-432e-a91a-4e9bec7f2c43'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:08.458470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d7e7f08a4f4c848204193b02f751c0', 'request_id': 'req-98146ac1-d17d-432e-a91a-4e9bec7f2c43'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:08.465040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d7e7f08a4f4c848204193b02f751c0', 'request_id': 'req-98146ac1-d17d-432e-a91a-4e9bec7f2c43'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:08.476751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] The token cache is being invalidate because project 55d7e7f08a4f4c848204193b02f751c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:08.477052 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:08.477681 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-98146ac1-d17d-432e-a91a-4e9bec7f2c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:08.479943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1347/2723] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:08 2022] DELETE /identity/v3/projects/55d7e7f08a4f4c848204193b02f751c0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:08.484729 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.486033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.486033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.505542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:08.506939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:08.553162 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:08.570208 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1377/2724] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:08.578269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.578558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.580529 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.599155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:08.600484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:08.609029 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1348/2725] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:08.613303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:08.623311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c936a-a9e5-4072-85aa-b66d5386ff7c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:08.627960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:08.628555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.628837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.629111 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.629757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:08.630088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:08.631664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:08.643197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1378/2726] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:08.647657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:08.657358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab42216-0b86-4a4b-ac68-ce9b935f3d20 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:08.662096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:08.662689 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.662966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.663233 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.663867 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:08.665801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:08.773776 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1349/2727] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:08 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:08.783858 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:08.795119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf959ef-8e9a-43af-827d-bd4c74e26f94 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:08.851476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:08.852079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:08.852357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:08.852627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:08.853262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:08.855795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:08.984376 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1379/2728] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:08 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:08.988555 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:08.998841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3411b9-7229-40e5-80b3-82105ca77b79 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:09.003880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:09.007653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.007653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.007653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.007653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:09.007653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:09.018710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1350/2729] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:09.022836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:09.034161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae44c29-332d-46d7-b215-35e14f12074d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:09.039276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:09.040017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.040302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.040669 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.041355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:09.042978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990842b-3549-46d9-a30e-b7a625ad0ee3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:09.054178 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1380/2730] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:09.059677 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:09.069735 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e0383-6518-4422-beaf-4a5db8fa9ba4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:09.075064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:09.075711 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.075992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.076269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] PATH_INFO: `/v3/projects/420ff8c80d8542c7818d2c7caafb4eb7/users/39ff00872fcf4a428520f1be1568e5c6/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.091639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420ff8c80d8542c7818d2c7caafb4eb7, user_id=39ff00872fcf4a428520f1be1568e5c6, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:09.094328 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:09.184967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1351/2731] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:09 2022] PUT /identity/v3/projects/420ff8c80d8542c7818d2c7caafb4eb7/users/39ff00872fcf4a428520f1be1568e5c6/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:09.191031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.191315 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.191585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.211399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:09.213470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:09.282247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1381/2732] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:09.292107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:09.293926 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:09.354763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637c8b7-45dd-4104-b690-56d33d94fc02 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:09.359187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:09.360859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.360859 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.361203 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.372505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:09.374032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30d2d9-c6cc-4ec8-8f85-c73acf42783a service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:09.386806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1352/2733] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:09.537030 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:09.538487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:09.604756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:09.637272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d6155-077e-4767-95f9-1ccad5ffb212 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:09.644486 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:09.650834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.650834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.650834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.650834 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:09.664492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:09.667204 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:09.680817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1382/2734] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:11.075891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:11.081481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:11.081481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:11.090715 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:11.106197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] MFA Rules not processed for user `39ff00872fcf4a428520f1be1568e5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:11.107438 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:11.175420 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1353/2735] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:11.186488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:11.187951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:11.247370 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387123d6-4c7d-44f0-b7f7-a9eb00f7eb77 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:11.250847 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:11.251439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:11.251712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:11.251980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:11.260469 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:11.261863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b94a3-c1d3-41ae-9404-ae747e747f92 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:11.270658 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1383/2736] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:11.957268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:11.958813 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:11.973969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e54a2b-d2ee-46d5-a3b3-9107697daa35 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:11.979209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:11.979836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:11.980115 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:11.980385 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] PATH_INFO: `/v3/users/6a332258680f4c7ea105432cad23deb6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:11.988965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a332258680f4c7ea105432cad23deb6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:11.988965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:12.279091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a332258680f4c7ea105432cad23deb6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:12.292009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a332258680f4c7ea105432cad23deb6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:12.298747 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a332258680f4c7ea105432cad23deb6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:12.372132 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a332258680f4c7ea105432cad23deb6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:12.372685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a332258680f4c7ea105432cad23deb6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:12.381628 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1354/2737] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:11 2022] DELETE /identity/v3/users/6a332258680f4c7ea105432cad23deb6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:12.386385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:12.386799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:12.387035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:12.406403 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:12.407614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:12.466005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1384/2738] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:12.475949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:12.477387 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:12.577775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886fb00-8b4f-408f-87bf-499fefcedf22 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:12.581295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:12.581888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:12.582188 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:12.582456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:12.591535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:12.592936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e1fc-8511-44a5-b599-8a30f8ab350b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:12.602100 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1355/2739] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:13.019907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.021331 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.034312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbb01b-285c-4892-8151-13b7a914befa None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.039983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.040689 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.041012 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.041848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] PATH_INFO: `/v3/projects/6ce35766a30e43d0a87e281fd0fdec81` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.043269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce35766a30e43d0a87e281fd0fdec81)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.045390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.206685 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce35766a30e43d0a87e281fd0fdec81', 'request_id': 'req-b140543c-12c1-4338-b5cc-1de09f99651f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:13.213836 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce35766a30e43d0a87e281fd0fdec81', 'request_id': 'req-b140543c-12c1-4338-b5cc-1de09f99651f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:13.219364 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce35766a30e43d0a87e281fd0fdec81', 'request_id': 'req-b140543c-12c1-4338-b5cc-1de09f99651f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:13.274558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] The token cache is being invalidate because project 6ce35766a30e43d0a87e281fd0fdec81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:13.274764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:13.275940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b140543c-12c1-4338-b5cc-1de09f99651f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:13.277532 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1385/2740] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:13 2022] DELETE /identity/v3/projects/6ce35766a30e43d0a87e281fd0fdec81 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:13.282495 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13813372-6634-46db-b24e-36cc160f61ab None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.283002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13813372-6634-46db-b24e-36cc160f61ab None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.283148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13813372-6634-46db-b24e-36cc160f61ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.302789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-13813372-6634-46db-b24e-36cc160f61ab None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:13.303986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-13813372-6634-46db-b24e-36cc160f61ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.309607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:13.362673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1356/2741] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:13.370641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.370925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.371197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.379898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:13.391718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:13.392928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.401761 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1386/2742] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:13.407411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13813372-6634-46db-b24e-36cc160f61ab None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.418154 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13813372-6634-46db-b24e-36cc160f61ab None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.422672 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.423267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.425464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.425464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.425464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:13.425464 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.426351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.437680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1357/2743] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:13.442344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.452839 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c00e3-ee08-48af-8aaa-cc0e02ae052d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.457585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.458219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.458505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.459837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.459837 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.461313 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.539496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1387/2744] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:13.543526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.553375 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea6b8c-c34e-4c9d-acca-d1f880b3058b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.610626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.611230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.611510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.611784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.612416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.614348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.705483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1358/2745] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:13.709863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.721371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f1433-2658-475b-a543-fb4de3f3bddd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.725982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.726637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.726918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.727193 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.727853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.730085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.740079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1388/2746] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:13.743980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.754684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae2645-2618-470a-81a7-a828d74963f6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.759354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.759976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.760255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.760527 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.761176 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.762795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49a94e-4e87-42ab-a650-c53d59c04e83 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.774182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1359/2747] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:13.778677 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.788405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca85e-7221-49ea-8892-dfb4e0c2b162 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:13.793607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:13.794213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.794488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.794755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] PATH_INFO: `/v3/projects/6788f3460c514b4693b0f435fddf1b73/users/424558d88ec643c58ee829baf265a5a8/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.808994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6788f3460c514b4693b0f435fddf1b73, user_id=424558d88ec643c58ee829baf265a5a8, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:13.811816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:13.892129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1389/2748] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:13 2022] PUT /identity/v3/projects/6788f3460c514b4693b0f435fddf1b73/users/424558d88ec643c58ee829baf265a5a8/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:58:13.903021 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.903303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.903577 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.931727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] MFA Rules not processed for user `424558d88ec643c58ee829baf265a5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:13.932945 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.995502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1360/2749] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:14.000175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.060253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e55e-1238-46ae-8ed4-b84b36e7c025 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.064776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.065373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.065651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.065954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.066611 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.068488 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561865-68ce-4670-a0dd-774c10ed18b9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.158965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1390/2750] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:14.162406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.174409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13d96-8ffa-40da-a92f-3ebf5f7f614b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.179419 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:14.207117 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.207395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.207668 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.229216 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] MFA Rules not processed for user `39ff00872fcf4a428520f1be1568e5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:14.230530 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:14.244991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.245594 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.245870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.246172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.246805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.248708 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.287779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1391/2751] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:14.344886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1361/2752] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:14 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:14.349243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.360966 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d79e6-8844-4b23-af78-6022c41afbcf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.369390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.370042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.370999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.370999 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.371371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.372908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.383309 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1392/2753] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:14.388377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.399178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5075c-692d-40b3-8ad1-f4eb674845b3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.404733 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.405441 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.405780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.406160 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.406879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.408545 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.420425 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1362/2754] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:14.424735 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.435772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b48e-5ef0-4c08-a8f6-15046e9284e1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.440098 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.440850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.441211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.441547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] PATH_INFO: `/v3/projects/fa1fe8a3d5c8492a96219ec2b69fa7f6/users/2679b29cb37c4851bdaa58d2673b67bd/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.457727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1fe8a3d5c8492a96219ec2b69fa7f6, user_id=2679b29cb37c4851bdaa58d2673b67bd, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.460462 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f2244-96d0-498c-a429-496237a4ced5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.464021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:14.535660 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.537405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:14.550552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1393/2755] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:14 2022] PUT /identity/v3/projects/fa1fe8a3d5c8492a96219ec2b69fa7f6/users/2679b29cb37c4851bdaa58d2673b67bd/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:14.600288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561584e-7585-47e1-a906-be63abf3fe8d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.618255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.623922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.623922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.646323 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] MFA Rules not processed for user `2679b29cb37c4851bdaa58d2673b67bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:14.647554 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:14.650982 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.651620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.651899 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.652172 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.654122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:14.663722 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.665142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.673414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1363/2756] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:58:14.714158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1394/2757] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:14.718655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.764232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:14.782128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518e1d-d967-45e5-a4fa-a6f0c7575e16 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.786684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.789709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.789709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.789709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.789709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.790408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.929384 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1364/2758] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:14 2022] POST /identity/v3/projects => generated 375 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:14.933732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:14.944539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fd1d-ea39-4221-8ca3-5a5ec74c7d5d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.004897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.005501 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.005779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.006047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.006705 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.008845 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.144804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1395/2759] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:14 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:15.148846 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:15.160866 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77348b3-a4f7-4299-9671-fff0ffc19a54 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.165679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.166320 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.170528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.170528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.170528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.170528 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.183914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1365/2760] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:15.185399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:15.207405 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7308b4-aafe-476b-a1a6-443a5098ed2b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.218928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.222600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.222600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.222600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] PATH_INFO: `/v3/projects/e799c3d70d5740fba10504f5590791a9/users/3019b917277143a5a918a2b0732275b0/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.236730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e799c3d70d5740fba10504f5590791a9, user_id=3019b917277143a5a918a2b0732275b0, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.239572 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.303890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1396/2761] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:15 2022] PUT /identity/v3/projects/e799c3d70d5740fba10504f5590791a9/users/3019b917277143a5a918a2b0732275b0/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:15.308591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:15.318504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532081c-ea0c-4854-85d9-9035ccf92836 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.373606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.374316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.374544 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.374841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.375494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.377075 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953c0aa-64c7-4d86-b4ed-2542c39a6feb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.388510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1366/2762] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:15.392800 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:15.403247 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180b59-d2ba-4f96-8676-90bd4d2818be admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.407903 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.408539 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.408812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.409085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] PATH_INFO: `/v3/projects/e799c3d70d5740fba10504f5590791a9/users/3019b917277143a5a918a2b0732275b0/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.411027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e799c3d70d5740fba10504f5590791a9, user_id=3019b917277143a5a918a2b0732275b0, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.413593 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.425551 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1397/2763] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:15 2022] PUT /identity/v3/projects/e799c3d70d5740fba10504f5590791a9/users/3019b917277143a5a918a2b0732275b0/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:15.440136 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.440490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.440782 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.469020 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] MFA Rules not processed for user `3019b917277143a5a918a2b0732275b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:15.470260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:15.538682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1367/2764] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:15.549768 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:15.551436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:15.594443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:15.610899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e28bfb-5d2d-467f-bf56-aeb53503e425 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:15.615000 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:15.615631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:15.619505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:15.619505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:15.672952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:15.674326 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:15.683842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1398/2765] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:16.916643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:16.918081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:16.962315 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:16.994435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69646354-6909-43e1-87a3-7c77c10158fd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:17.000311 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:17.001095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:17.001748 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:17.002096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:17.003836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:17.083807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:17.085411 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:17.110727 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1368/2766] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:19.095387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:19.104253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e1fb7-3d74-485a-8a51-07fd4c19ca9f service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:19.108143 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:19.108936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:19.109487 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:19.109778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:19.167574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:19.168919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:19.178184 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1399/2767] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:21.026427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:21.035707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba4c9c-e562-4e54-ae72-b1bd9c9443d4 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:21.039911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:21.040534 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:21.041647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:21.041647 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:21.050879 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:21.052432 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d78367-9ca6-45a5-9f61-2f84d10f0444 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:21.062569 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1369/2768] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:27.127783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:27.192561 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0a366-51d4-40ee-a800-27e57490d67a service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:27.197161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:27.197782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:27.198076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:27.198360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] PATH_INFO: `/v3/users/424558d88ec643c58ee829baf265a5a8` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:27.199511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] RBAC: Authorizing `identity:delete_user(user_id=424558d88ec643c58ee829baf265a5a8)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:27.201435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:27.379994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:27.392569 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424558d88ec643c58ee829baf265a5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:27.407592 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424558d88ec643c58ee829baf265a5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:27.414432 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424558d88ec643c58ee829baf265a5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:27.416958 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424558d88ec643c58ee829baf265a5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:27.498294 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424558d88ec643c58ee829baf265a5a8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:27.505363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1400/2769] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:27 2022] DELETE /identity/v3/users/424558d88ec643c58ee829baf265a5a8 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:27.511034 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:27.511947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:27.511947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:27.532753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:27.534182 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:27.597422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1370/2770] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:27.608282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:27.609655 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:27.669528 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3354f-9562-4765-9a0c-eaf1b4da002d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:27.673203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:27.673813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:27.674244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:27.674529 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:27.683760 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:27.685172 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:27.694741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1401/2771] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:28.180058 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:28.182349 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:28.195267 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d6463-3b9a-44cc-bd7d-7ff9db98438f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:28.208993 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:28.209433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:28.209968 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:28.211257 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] PATH_INFO: `/v3/projects/6788f3460c514b4693b0f435fddf1b73` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:28.212755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6788f3460c514b4693b0f435fddf1b73)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:28.215177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:28.315024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:28.328988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6788f3460c514b4693b0f435fddf1b73', 'request_id': 'req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.395477 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6788f3460c514b4693b0f435fddf1b73', 'request_id': 'req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.401589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6788f3460c514b4693b0f435fddf1b73', 'request_id': 'req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.413338 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] The token cache is being invalidate because project 6788f3460c514b4693b0f435fddf1b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:28.413624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.414270 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.416285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1371/2772] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:28 2022] DELETE /identity/v3/projects/6788f3460c514b4693b0f435fddf1b73 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:28.420361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:28.430230 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60027648-23b4-4c07-8d1b-58553ea79e81 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:28.487448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:28.488066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:28.488343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:28.488619 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] PATH_INFO: `/v3/users/2679b29cb37c4851bdaa58d2673b67bd` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:28.489737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2679b29cb37c4851bdaa58d2673b67bd)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:28.491663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:28.649007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2679b29cb37c4851bdaa58d2673b67bd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.654480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:28.661454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2679b29cb37c4851bdaa58d2673b67bd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.668721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2679b29cb37c4851bdaa58d2673b67bd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.669390 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2679b29cb37c4851bdaa58d2673b67bd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.709259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2679b29cb37c4851bdaa58d2673b67bd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:28.715406 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1402/2773] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:28 2022] DELETE /identity/v3/users/2679b29cb37c4851bdaa58d2673b67bd => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:29.437869 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:29.439508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:29.453399 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25b8-16d6-4e9c-89c4-c50510c5fd65 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:29.510394 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:29.510953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:29.511223 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:29.511497 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] PATH_INFO: `/v3/projects/fa1fe8a3d5c8492a96219ec2b69fa7f6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:29.512615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1fe8a3d5c8492a96219ec2b69fa7f6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:29.514549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:29.624965 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1fe8a3d5c8492a96219ec2b69fa7f6', 'request_id': 'req-39035bd4-16ab-419f-bd52-80bc02be08b4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:29.726481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1fe8a3d5c8492a96219ec2b69fa7f6', 'request_id': 'req-39035bd4-16ab-419f-bd52-80bc02be08b4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:29.732460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1fe8a3d5c8492a96219ec2b69fa7f6', 'request_id': 'req-39035bd4-16ab-419f-bd52-80bc02be08b4'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:29.744891 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] The token cache is being invalidate because project fa1fe8a3d5c8492a96219ec2b69fa7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:29.745177 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:29.745823 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:29.747367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1372/2774] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:29 2022] DELETE /identity/v3/projects/fa1fe8a3d5c8492a96219ec2b69fa7f6 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:29.753802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:29.753802 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:29.766693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ef80-da1e-4b7c-9cf9-859ba5c8dcc2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:29.822670 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:29.823327 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:29.823608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:29.823881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] PATH_INFO: `/v3/users/3019b917277143a5a918a2b0732275b0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:29.824997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3019b917277143a5a918a2b0732275b0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:29.826960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.009060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3019b917277143a5a918a2b0732275b0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.021181 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3019b917277143a5a918a2b0732275b0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.028071 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3019b917277143a5a918a2b0732275b0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.028833 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3019b917277143a5a918a2b0732275b0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.104158 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6deeece7-c54c-4224-8ee6-299c3f86ec5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3019b917277143a5a918a2b0732275b0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.110229 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1403/2775] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:29 2022] DELETE /identity/v3/users/3019b917277143a5a918a2b0732275b0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:58:30.514413 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.515933 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.530201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39035bd4-16ab-419f-bd52-80bc02be08b4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.574707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:30.586264 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.593372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.593372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.593372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] PATH_INFO: `/v3/projects/e799c3d70d5740fba10504f5590791a9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.593372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e799c3d70d5740fba10504f5590791a9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.593372 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.703805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e799c3d70d5740fba10504f5590791a9', 'request_id': 'req-b38eafeb-b5e6-4311-9992-da922bf1e62f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.777415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e799c3d70d5740fba10504f5590791a9', 'request_id': 'req-b38eafeb-b5e6-4311-9992-da922bf1e62f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.783256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e799c3d70d5740fba10504f5590791a9', 'request_id': 'req-b38eafeb-b5e6-4311-9992-da922bf1e62f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.794301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] The token cache is being invalidate because project e799c3d70d5740fba10504f5590791a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:30.798786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.798786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b38eafeb-b5e6-4311-9992-da922bf1e62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:30.798786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1373/2776] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:30 2022] DELETE /identity/v3/projects/e799c3d70d5740fba10504f5590791a9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:30.803243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664433-0f36-42af-a445-155a045407f4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.803733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664433-0f36-42af-a445-155a045407f4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.803843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664433-0f36-42af-a445-155a045407f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.823522 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-75664433-0f36-42af-a445-155a045407f4 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:30.827520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-75664433-0f36-42af-a445-155a045407f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.868261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:30.890187 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1404/2777] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:30.899393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.899680 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.899949 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.920026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:30.921222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.928986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1374/2778] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:30.933264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75664433-0f36-42af-a445-155a045407f4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.943051 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75664433-0f36-42af-a445-155a045407f4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.948253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.948850 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.949132 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.949407 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.950078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:30.950387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.951963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.963947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1405/2779] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:30.968472 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.977801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dab1c3-d084-4c09-bcc4-f69c88237346 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.982671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.988841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.988841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.988841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.988841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.988841 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.087719 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1375/2780] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:30 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:31.092029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:31.102038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3477e28-21e0-4706-a1cf-3158fc99d7a6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:31.157616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:31.158264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.158540 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.158807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.159434 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:31.161306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.269687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1406/2781] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:31 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:31.273744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:31.284110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b3503-27e7-42e5-a68a-269db6723d51 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:31.288705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:31.289295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.289572 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.289951 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.290641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:31.292213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.301904 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1376/2782] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:31.305817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:31.316519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938310-2fe5-433a-be8e-ed5714c20ddf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:31.321333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:31.321931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.322816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.323092 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.323734 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:31.325361 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75bc6-4147-45dc-8042-7710b9841e46 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.335855 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1407/2783] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:31.340202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:31.349935 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4866-3ab4-4b6d-a719-b77faae91215 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:31.354470 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:31.355102 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.355379 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.355652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] PATH_INFO: `/v3/projects/d19dac54b1ce4571b533178a1dbc4c39/users/f5a780d912d64a7f9b8238f8021f7b24/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.370456 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19dac54b1ce4571b533178a1dbc4c39, user_id=f5a780d912d64a7f9b8238f8021f7b24, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:31.373089 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.510353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1377/2784] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:31 2022] PUT /identity/v3/projects/d19dac54b1ce4571b533178a1dbc4c39/users/f5a780d912d64a7f9b8238f8021f7b24/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:31.515631 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195e8b-5837-4493-b208-ef56e7722719 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.515947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195e8b-5837-4493-b208-ef56e7722719 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.516225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195e8b-5837-4493-b208-ef56e7722719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.535436 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bf195e8b-5837-4493-b208-ef56e7722719 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:31.536635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bf195e8b-5837-4493-b208-ef56e7722719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:31.596102 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1408/2785] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:31.606168 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:31.607864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:31.666884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34df09-0459-4bbe-9254-121bc2a3b100 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:31.671081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:31.671781 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:31.672064 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:31.672334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:31.682624 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:31.684043 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:31.693369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1378/2786] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:37.415113 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa629e-e009-40ff-9967-947d75657e46 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.415737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa629e-e009-40ff-9967-947d75657e46 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.415737 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa629e-e009-40ff-9967-947d75657e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.484815 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-effa629e-e009-40ff-9967-947d75657e46 None None] MFA Rules not processed for user `f5a780d912d64a7f9b8238f8021f7b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:37.486089 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-effa629e-e009-40ff-9967-947d75657e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:37.571899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:37.590312 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1409/2787] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:37.592469 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:37.603109 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587dd45-7486-4320-86d3-488b947950cf service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:37.608125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:37.608736 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.609014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.609291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.609932 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:37.615240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:37.634221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:37.706725 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1379/2788] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:37 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:37.710810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa629e-e009-40ff-9967-947d75657e46 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:37.723243 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa629e-e009-40ff-9967-947d75657e46 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:37.784422 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:37.788692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.788692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.788692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.788692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:37.788692 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:37.898583 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1410/2789] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:37 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:37.902785 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:37.917709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb21f2b-c377-4050-8e17-f17e9a1b8e38 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:37.922475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:37.923157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.923430 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.923697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.924326 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:37.925952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:37.938327 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1380/2790] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:37.941957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:37.952371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447e98-bb3c-4ae4-aa67-2cedfe5e8180 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:37.957917 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:37.958601 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.958878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.959150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] PATH_INFO: `/v3/projects/570155e09fd5431384350cd10f5f8ec8/users/de9f32a80a25484d90856dce4f4d6114/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.973813 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570155e09fd5431384350cd10f5f8ec8, user_id=de9f32a80a25484d90856dce4f4d6114, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:37.976463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:38.083067 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1411/2791] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:37 2022] PUT /identity/v3/projects/570155e09fd5431384350cd10f5f8ec8/users/de9f32a80a25484d90856dce4f4d6114/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:38.087705 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:38.114141 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0df39f-021a-4660-a856-8298c91ba3c0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:38.158242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:38.158887 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:38.159158 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:38.159422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:38.160112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:38.161777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9eb0d2-bb1c-4eab-be5c-f03debcca2ca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:38.172720 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1381/2792] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:38.176816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:38.187828 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a236d8b-45ca-432c-b1a1-5b0248d95ff2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:38.193082 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:38.193759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:38.194040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:38.194353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] PATH_INFO: `/v3/projects/570155e09fd5431384350cd10f5f8ec8/users/de9f32a80a25484d90856dce4f4d6114/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:38.196357 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=570155e09fd5431384350cd10f5f8ec8, user_id=de9f32a80a25484d90856dce4f4d6114, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:38.199962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:38.211507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1412/2793] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:38 2022] PUT /identity/v3/projects/570155e09fd5431384350cd10f5f8ec8/users/de9f32a80a25484d90856dce4f4d6114/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:45.236975 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3d2e-383f-44ef-9b9d-b0c725d728b6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.237966 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3d2e-383f-44ef-9b9d-b0c725d728b6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.238536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3d2e-383f-44ef-9b9d-b0c725d728b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.287518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1e0a3d2e-383f-44ef-9b9d-b0c725d728b6 None None] MFA Rules not processed for user `de9f32a80a25484d90856dce4f4d6114`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:45.288796 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1e0a3d2e-383f-44ef-9b9d-b0c725d728b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:45.394342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1382/2794] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:45.409520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:45.412061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:45.527006 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af249c9-aa25-4249-9a07-0eb13e8faa9f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:45.535005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:45.535728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.536085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.536427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.586656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:45.631963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:45.633804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:45.657088 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1413/2795] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:48.638798 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:48.639533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:48.639533 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:48.658503 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] MFA Rules not processed for user `f5a780d912d64a7f9b8238f8021f7b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:48.659911 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:48.669261 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1383/2796] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:50.931496 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:50.996401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b18b8-4454-4940-b672-2cac136148cd service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:50.997194 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:50.997878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:50.998192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:50.998467 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.000277 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:51.068500 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:51.069951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:51.080201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1414/2797] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:54.522947 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.524653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:54.594955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.600816 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc98b2c-618f-4f15-9173-e748378a7b31 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.606650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.607269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.607549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.607824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.609630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:54.621415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.623118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.632925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1384/2798] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:54.680778 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.682249 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:54.694517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e062ef-1470-4651-af32-96f464fbc295 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.699794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.700416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.700696 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.700969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] PATH_INFO: `/v3/users/b193d0924fb448bfac2027034cb8dfe5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.702110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b193d0924fb448bfac2027034cb8dfe5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.704009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.917954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b193d0924fb448bfac2027034cb8dfe5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:54.930379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b193d0924fb448bfac2027034cb8dfe5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:54.937324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b193d0924fb448bfac2027034cb8dfe5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:54.938014 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b193d0924fb448bfac2027034cb8dfe5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.018024 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23a83e40-5318-4648-8e31-c9428f69fbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b193d0924fb448bfac2027034cb8dfe5'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.019741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:55.025461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1415/2799] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:58:54 2022] DELETE /identity/v3/users/b193d0924fb448bfac2027034cb8dfe5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:55.318382 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:55.330078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c366d-c8b7-4ef6-bc69-4c3001daa142 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:55.413028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.413669 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.414118 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.414418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] PATH_INFO: `/v3/projects/f02e440a52824687b246cb75b4f135cd` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.415570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02e440a52824687b246cb75b4f135cd)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.418106 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:55.516071 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02e440a52824687b246cb75b4f135cd', 'request_id': 'req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.618583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02e440a52824687b246cb75b4f135cd', 'request_id': 'req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.626767 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02e440a52824687b246cb75b4f135cd', 'request_id': 'req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.640697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] The token cache is being invalidate because project f02e440a52824687b246cb75b4f135cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:58:55.640990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.641663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b2f1cefb-1182-41a6-9790-c7e7cc8f04e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:58:55.643183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1385/2800] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:58:55 2022] DELETE /identity/v3/projects/f02e440a52824687b246cb75b4f135cd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:55.648026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.648251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.648525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.670159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:55.671378 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:55.733145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1416/2801] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:55.741151 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.741498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.741721 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.761499 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:55.762755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:55.770840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1386/2802] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:55.774886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:55.788738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908c0b3-6185-4e17-8c6a-5d0011bbe6cc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:55.794936 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.795579 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.795995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.796282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.796942 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:55.797239 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.798890 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:55.811884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1417/2803] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:55.816329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:55.825934 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518914d-f080-489b-a7dd-cb472ec0e554 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:55.830960 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.831800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.832078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.832344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.833211 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.835631 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:55.922918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1387/2804] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:55.927090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:55.937416 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d771-a6d8-48e0-a08a-7feab6260bcb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:55.995553 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.996164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.996441 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.996712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.997355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.999291 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.122079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1418/2805] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/users => generated 279 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:56.125916 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.136655 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d3805-f810-4d0f-9711-14d65a0dd1b4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.141710 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.148716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.148716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.148716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.148716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.148716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.157757 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1388/2806] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:56.161956 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.172340 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc9806-a095-4af6-84f7-382197a03aad admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.176902 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.177495 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.177774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.178045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.178709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.180276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37133b-3dbb-4c02-9744-2e02eef8c2fa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.190911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1419/2807] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:56.194790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.204921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b743f-4adf-443b-b48b-ecbb90979186 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.209373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.210001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.210304 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.210573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] PATH_INFO: `/v3/projects/cb59d9c977b2465fb9734d6c22a086e0/users/6b145e58744c4f1bb6f11a53a381edc0/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.225305 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb59d9c977b2465fb9734d6c22a086e0, user_id=6b145e58744c4f1bb6f11a53a381edc0, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.227950 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.321517 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1389/2808] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:56 2022] PUT /identity/v3/projects/cb59d9c977b2465fb9734d6c22a086e0/users/6b145e58744c4f1bb6f11a53a381edc0/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:56.333605 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.334307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.334613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.368732 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] MFA Rules not processed for user `6b145e58744c4f1bb6f11a53a381edc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:56.369983 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:56.429311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.441276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1420/2809] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:56.446045 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.466296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.509126 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc7b6e-752e-45c0-818a-af3fbf8d4940 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.514440 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.515358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.515358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.515581 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.516169 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.518072 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.624817 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1390/2810] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:58:56 2022] POST /identity/v3/projects => generated 357 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:56.628875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.641269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d38a50-ebfe-44bb-982e-e0d09daddb71 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.705609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.706171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.706515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.706787 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.707410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.709297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.807170 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1421/2811] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:58:56 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:56.811753 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.822746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d208f-bbba-44ef-97c7-440ed03bed63 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.828294 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.828925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.829202 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.829471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.830181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.831789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.845612 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1391/2812] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:58:56.849741 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.860199 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cc27-093d-475c-a43c-844de75a47f1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.864794 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.865433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.865709 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.865977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] PATH_INFO: `/v3/projects/48deb473ee19421b9b6be61a07d0e268/users/0965851deed145c5960380a3e7930437/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.881483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=48deb473ee19421b9b6be61a07d0e268, user_id=0965851deed145c5960380a3e7930437, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.884164 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.961687 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1422/2813] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:56 2022] PUT /identity/v3/projects/48deb473ee19421b9b6be61a07d0e268/users/0965851deed145c5960380a3e7930437/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:56.966117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.976282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f7a11-1fd6-4109-9c74-3efbb13f3453 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.035275 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.035885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.036166 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.036436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.037099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.038766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd96558-38f8-40c9-8a5e-e58d54c0a184 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.048682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1392/2814] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:58:57.052881 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.063686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7822d-a432-40ea-a383-4de00b4b8aea admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.068190 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.068825 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.069099 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.069365 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] PATH_INFO: `/v3/projects/48deb473ee19421b9b6be61a07d0e268/users/0965851deed145c5960380a3e7930437/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.071270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48deb473ee19421b9b6be61a07d0e268, user_id=0965851deed145c5960380a3e7930437, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.073851 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.083765 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1423/2815] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:58:57 2022] PUT /identity/v3/projects/48deb473ee19421b9b6be61a07d0e268/users/0965851deed145c5960380a3e7930437/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:57.096003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.096515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.096794 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.125134 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] MFA Rules not processed for user `0965851deed145c5960380a3e7930437`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:57.126424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:57.196333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1393/2816] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:57.207445 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.209209 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:57.266301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a5477-f765-4167-a084-c4497ad2ad2d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.269944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.270567 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.270848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.271120 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.280384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.281783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.291586 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1424/2817] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:57.517418 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.528423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254e5a1-1c00-47e4-b692-61c33eb9b790 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.532225 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.532845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.533128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.533515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.542773 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:57.592650 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.594129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37383-f9e0-441b-b2d6-9ce2e24d3e12 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.603619 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1394/2818] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:59:07.286961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.635488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=23 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:26.678541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:26.740302 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675dfaf-0798-439e-b89a-aa56b44b569b service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:26.744873 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:26.745506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.745770 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.746045 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] PATH_INFO: `/v3/users/6b145e58744c4f1bb6f11a53a381edc0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.747259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b145e58744c4f1bb6f11a53a381edc0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:26.749213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:26.900597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b145e58744c4f1bb6f11a53a381edc0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:26.912615 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b145e58744c4f1bb6f11a53a381edc0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:26.919563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b145e58744c4f1bb6f11a53a381edc0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:26.920351 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b145e58744c4f1bb6f11a53a381edc0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:26.995251 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b145e58744c4f1bb6f11a53a381edc0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.001543 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1425/2819] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:26 2022] DELETE /identity/v3/users/6b145e58744c4f1bb6f11a53a381edc0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:27.006164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.006664 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.006786 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.025986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:27.027195 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.086849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1395/2820] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:27.096730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.098151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.154179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252bb8-fb91-495c-8e84-6b28d4d1a54c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.157713 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.158338 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.158614 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.158884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.167925 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.169305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:27.178750 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1426/2821] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:59:27.614076 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.615511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.634127 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011d237-e6db-43ea-a319-dfda6c94dd4d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.645663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.646319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.646589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.646854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] PATH_INFO: `/v3/projects/cb59d9c977b2465fb9734d6c22a086e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.647953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb59d9c977b2465fb9734d6c22a086e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.650591 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:27.737836 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb59d9c977b2465fb9734d6c22a086e0', 'request_id': 'req-97e3c683-b11c-4226-be40-483bdc8b90ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.783416 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb59d9c977b2465fb9734d6c22a086e0', 'request_id': 'req-97e3c683-b11c-4226-be40-483bdc8b90ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.789194 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb59d9c977b2465fb9734d6c22a086e0', 'request_id': 'req-97e3c683-b11c-4226-be40-483bdc8b90ae'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.800218 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] The token cache is being invalidate because project cb59d9c977b2465fb9734d6c22a086e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:27.800504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.801129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:27.802623 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1396/2822] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:27 2022] DELETE /identity/v3/projects/cb59d9c977b2465fb9734d6c22a086e0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:27.807259 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.816930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba3c2d-b76d-46a3-9dbc-d0754cef40e5 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.835453 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.873798 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.874455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.874736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.875017 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] PATH_INFO: `/v3/users/0965851deed145c5960380a3e7930437` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.876135 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0965851deed145c5960380a3e7930437)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.878028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.084054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0965851deed145c5960380a3e7930437'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.096298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0965851deed145c5960380a3e7930437'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.103882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0965851deed145c5960380a3e7930437'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.104609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0965851deed145c5960380a3e7930437'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.161358 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0d78a0dd-704f-4a63-ac7b-ae8ab1ca7aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0965851deed145c5960380a3e7930437'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.167953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1427/2823] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:27 2022] DELETE /identity/v3/users/0965851deed145c5960380a3e7930437 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:28.591096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.592573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:28.606209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c683-b11c-4226-be40-483bdc8b90ae admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.665963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.666617 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.666896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.667165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] PATH_INFO: `/v3/projects/48deb473ee19421b9b6be61a07d0e268` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.668614 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48deb473ee19421b9b6be61a07d0e268)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.670550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.767847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:28.783629 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48deb473ee19421b9b6be61a07d0e268', 'request_id': 'req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.852568 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48deb473ee19421b9b6be61a07d0e268', 'request_id': 'req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.858298 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48deb473ee19421b9b6be61a07d0e268', 'request_id': 'req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.870303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] The token cache is being invalidate because project 48deb473ee19421b9b6be61a07d0e268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:28.874245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.874245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8d40d623-5d44-440a-8b21-7b44b1a8fcc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:28.874245 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1397/2824] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:28 2022] DELETE /identity/v3/projects/48deb473ee19421b9b6be61a07d0e268 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:28.878379 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.878897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.879023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.898195 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:28.899397 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:28.958129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1428/2825] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:28.966232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.966512 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.966783 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.986948 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:28.988156 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:28.996659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1398/2826] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:29.000756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.010303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95894a-8cfc-4da4-bb3b-7e9c5bd1819d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.014864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.015454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.015733 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.016007 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.016647 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:29.016951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.018563 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.029871 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1429/2827] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:29.034352 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.043980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbe9a-5516-4508-8825-533e710bf1a0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.048515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.049097 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.049373 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.049639 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.050296 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.052209 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.136355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1399/2828] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:29.140597 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.151196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ba675-31a0-4571-9d15-f3abd9cdfac9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.176620 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:29.208027 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.208630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.208911 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.209180 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.209817 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.211783 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.287285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1430/2829] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:29.291734 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.302096 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caae778-1f1a-4038-a856-0fedb7c70c57 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.307426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.308003 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.308319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.308597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.309260 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.310888 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.321299 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1400/2830] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:59:29.325179 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.335034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e726b13-f841-477c-98af-f46d544bd6df admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.340512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.341119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.341392 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.341660 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.342333 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.343951 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fba21-7e2e-4ac5-a21d-dab2e8a56560 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.353609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1431/2831] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:29.357597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.373303 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbebb-3a78-440c-86a4-14e99a4301e0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.378389 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.381812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.381812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.381812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] PATH_INFO: `/v3/projects/070c41380db44b30bbbab00a0b307f17/users/0784a6cc612e4155bd8dd1d358eb054b/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.395636 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=070c41380db44b30bbbab00a0b307f17, user_id=0784a6cc612e4155bd8dd1d358eb054b, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.398755 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.475980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1401/2832] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:29 2022] PUT /identity/v3/projects/070c41380db44b30bbbab00a0b307f17/users/0784a6cc612e4155bd8dd1d358eb054b/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:29.488090 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.488367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.488651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.516297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] MFA Rules not processed for user `0784a6cc612e4155bd8dd1d358eb054b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:29.519702 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:29.583023 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1432/2833] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:29.587873 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.657597 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaccd59-9776-4b88-ab7e-d4fdfba7f60d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.662741 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.663428 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.663980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.664329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.665316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.667369 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.797129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1402/2834] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:29.801124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.812439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274b050-f0f3-4158-ad2b-37a78cc1b607 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.869960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.870600 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.870897 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.871171 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.871820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.873722 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.944667 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1433/2835] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:29.949359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.960268 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdfc36-5a9b-444f-a39b-e50f08e1f05d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:29.965259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:29.965819 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.968216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.968216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.968216 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:29.968726 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:29.979564 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1403/2836] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:29.985556 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:29.996283 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d695d6-12cd-41e1-86fe-0f58829e617c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:30.001480 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:30.002163 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.002448 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.002726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] PATH_INFO: `/v3/projects/8fcc7973165a46cd91706b063e61402e/users/54427a9a1b1e42d2b86bd38047075b41/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.018391 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcc7973165a46cd91706b063e61402e, user_id=54427a9a1b1e42d2b86bd38047075b41, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:30.021020 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:30.064219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1434/2837] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:29 2022] PUT /identity/v3/projects/8fcc7973165a46cd91706b063e61402e/users/54427a9a1b1e42d2b86bd38047075b41/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:30.068269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:30.079483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b3dca-bad7-4e95-a167-ee50743e0295 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:30.101119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:30.139685 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:30.140666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.140942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.141212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.141855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:30.143484 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fada3-d37a-47f6-b054-d81a42b2e4eb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:30.159206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1404/2838] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:30.160730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:30.166444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:30.174974 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994fe65-6026-44dd-9e4c-246f54649fa4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:30.176438 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:30.177084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.177363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.177636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] PATH_INFO: `/v3/projects/8fcc7973165a46cd91706b063e61402e/users/54427a9a1b1e42d2b86bd38047075b41/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.180077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcc7973165a46cd91706b063e61402e, user_id=54427a9a1b1e42d2b86bd38047075b41, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:30.182739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:30.198005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1435/2839] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:30 2022] PUT /identity/v3/projects/8fcc7973165a46cd91706b063e61402e/users/54427a9a1b1e42d2b86bd38047075b41/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:30.204675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.204986 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.205282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.233374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] MFA Rules not processed for user `54427a9a1b1e42d2b86bd38047075b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:30.234673 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.308772 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1405/2840] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:30.316369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:30.318156 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.378197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fecc36-0eca-4406-82ee-d7034b373a5f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:30.382131 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:30.382779 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.383057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.383332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.392688 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:30.394736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262db5d1-5a55-43ae-9477-651b9ddc80fd service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:30.403715 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1436/2841] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:33.025649 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.089222 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed80c0-9706-4aa8-8f1b-e92c467f6983 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.094183 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.094810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.095088 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.095421 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] PATH_INFO: `/v3/users/0784a6cc612e4155bd8dd1d358eb054b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.096552 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0784a6cc612e4155bd8dd1d358eb054b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.098644 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.250507 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0784a6cc612e4155bd8dd1d358eb054b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:33.251728 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.263914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0784a6cc612e4155bd8dd1d358eb054b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:33.271164 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0784a6cc612e4155bd8dd1d358eb054b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:33.332500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0784a6cc612e4155bd8dd1d358eb054b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:33.333129 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0784a6cc612e4155bd8dd1d358eb054b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:33.339659 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1406/2842] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:33 2022] DELETE /identity/v3/users/0784a6cc612e4155bd8dd1d358eb054b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:33.344228 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.344721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.344864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.364269 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:33.365483 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.425544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1437/2843] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:33.435882 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.437259 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.493185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29514a45-dbb1-44ec-8fe8-3c006ffaca86 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.496824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.497589 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.497864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.498165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.507288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.508775 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.518001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1407/2844] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:33.922360 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.923811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.936492 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9bd9-c820-47fe-ae06-36eb52282135 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.941298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.941945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.942319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.942679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] PATH_INFO: `/v3/projects/070c41380db44b30bbbab00a0b307f17` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.943853 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=070c41380db44b30bbbab00a0b307f17)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.946353 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.069295 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '070c41380db44b30bbbab00a0b307f17', 'request_id': 'req-82289a87-272b-444d-b5bb-aa884b6f0f7c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.076060 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '070c41380db44b30bbbab00a0b307f17', 'request_id': 'req-82289a87-272b-444d-b5bb-aa884b6f0f7c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.082728 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '070c41380db44b30bbbab00a0b307f17', 'request_id': 'req-82289a87-272b-444d-b5bb-aa884b6f0f7c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.189077 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] The token cache is being invalidate because project 070c41380db44b30bbbab00a0b307f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:34.189394 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.190009 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.191628 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1438/2845] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:33 2022] DELETE /identity/v3/projects/070c41380db44b30bbbab00a0b307f17 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:34.196242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.206642 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a3ea8-1766-42ef-8281-fbf15549d640 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.269192 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.269810 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.270111 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.270381 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] PATH_INFO: `/v3/users/54427a9a1b1e42d2b86bd38047075b41` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.271482 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54427a9a1b1e42d2b86bd38047075b41)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.274009 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.434936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54427a9a1b1e42d2b86bd38047075b41'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.447850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54427a9a1b1e42d2b86bd38047075b41'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.452408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.455116 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54427a9a1b1e42d2b86bd38047075b41'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.543609 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54427a9a1b1e42d2b86bd38047075b41'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.544235 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d9ba6417-1e9f-4e1a-87a6-549407f7c5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54427a9a1b1e42d2b86bd38047075b41'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:34.550353 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1408/2846] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:34 2022] DELETE /identity/v3/users/54427a9a1b1e42d2b86bd38047075b41 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:34.987844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.987844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:34.997632 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289a87-272b-444d-b5bb-aa884b6f0f7c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.011525 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.057866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.058607 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.058953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.059352 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] PATH_INFO: `/v3/projects/8fcc7973165a46cd91706b063e61402e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.060585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcc7973165a46cd91706b063e61402e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.062584 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.235882 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcc7973165a46cd91706b063e61402e', 'request_id': 'req-e0c84c01-043c-4b41-abfb-e340552adb26'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:35.242777 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcc7973165a46cd91706b063e61402e', 'request_id': 'req-e0c84c01-043c-4b41-abfb-e340552adb26'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:35.248263 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcc7973165a46cd91706b063e61402e', 'request_id': 'req-e0c84c01-043c-4b41-abfb-e340552adb26'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:35.360656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] The token cache is being invalidate because project 8fcc7973165a46cd91706b063e61402e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:35.360949 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:35.361559 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e0c84c01-043c-4b41-abfb-e340552adb26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:35.363101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1439/2847] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:34 2022] DELETE /identity/v3/projects/8fcc7973165a46cd91706b063e61402e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:35.368541 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.369039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.369159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.388907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:35.390121 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:35.448028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1409/2848] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:35.455820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.456101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.456374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.476021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:35.477401 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:35.485306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1440/2849] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:35.489458 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.499862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f80b13-b9e3-4364-b79a-520aee57bf59 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.504380 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.504976 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.505251 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.505522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.506654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:35.506654 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.508662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.524122 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1410/2850] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:35.525339 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.534474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb4dfd-96ea-4df1-ad2b-cb5e3a5e772c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.539591 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.540177 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.540454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.540721 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.541963 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.543868 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.662555 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1441/2851] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:35.664055 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.674535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e931-1c63-4317-9aff-006bf774b702 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.731339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.731938 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.732214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.732481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.733108 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.735032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.812219 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1411/2852] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:35.816266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.827065 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf3cfb-4699-43ab-b182-711f5ccb8d5a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.831698 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.832282 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.832587 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.832859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.833526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.835112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.845662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1442/2853] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:35.849536 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.859632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e38d6-a4c6-47b8-978a-b5660c53676a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.864255 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.864838 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.865117 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.865388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.866030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.867646 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627c84-2ba8-46f6-830e-3e52bbec42c8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.877970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1412/2854] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:35.881781 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.892043 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029774-fcab-4f93-8e27-816ee9b841bf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.896617 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.897242 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.897520 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.897795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] PATH_INFO: `/v3/projects/fd8a6048881949f3a3c766ea6e6f4900/users/9d3a89afb89d4b9f8733e14c6b6961b3/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.912919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8a6048881949f3a3c766ea6e6f4900, user_id=9d3a89afb89d4b9f8733e14c6b6961b3, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.915635 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.995771 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1443/2855] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:35 2022] PUT /identity/v3/projects/fd8a6048881949f3a3c766ea6e6f4900/users/9d3a89afb89d4b9f8733e14c6b6961b3/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:36.007196 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.008093 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.008374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.036295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] MFA Rules not processed for user `9d3a89afb89d4b9f8733e14c6b6961b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:36.037518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.078780 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.100730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1413/2856] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:36.105419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.150738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.167726 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3266d0b-1cdb-49e2-aa09-6582e3babd63 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.172463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.173061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.173343 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.173616 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.174276 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.176189 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.266409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1444/2857] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:36 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:36.270285 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.281556 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112179-1a47-4460-b236-88c2995bc6c0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.337348 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.337884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.338198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.338473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.339104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.341272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.430513 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1414/2858] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:36 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:36.434807 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.445463 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9ae3-5662-44b3-a6fb-06df96e12164 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.450203 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.450863 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.451145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.451444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.452069 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.453708 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.463681 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1445/2859] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:36.467995 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.478695 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac772a7-50a4-46cc-9709-288ddf54d95a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.483494 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.484161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.484442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.484709 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] PATH_INFO: `/v3/projects/90b0da67d6514dcd9b599510195ccbcc/users/24476420516245c3bd9b57eccb5dc7ea/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.501436 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b0da67d6514dcd9b599510195ccbcc, user_id=24476420516245c3bd9b57eccb5dc7ea, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.504112 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.595485 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1415/2860] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:36 2022] PUT /identity/v3/projects/90b0da67d6514dcd9b599510195ccbcc/users/24476420516245c3bd9b57eccb5dc7ea/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:36.599506 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.609899 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149e96b-f911-4955-a5e7-c1c03c8cc132 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.665304 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.665926 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.666238 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.666516 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.667192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.668782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3debf0-57bc-475e-b6e8-f0f17fb5bec6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.684262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1446/2861] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:36.685406 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.695745 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34ae7e-2e16-49bd-8d52-37c9b94422ab admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.700423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.701067 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.701342 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.701611 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] PATH_INFO: `/v3/projects/90b0da67d6514dcd9b599510195ccbcc/users/24476420516245c3bd9b57eccb5dc7ea/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.703523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b0da67d6514dcd9b599510195ccbcc, user_id=24476420516245c3bd9b57eccb5dc7ea, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.706161 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.717074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1416/2862] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:36 2022] PUT /identity/v3/projects/90b0da67d6514dcd9b599510195ccbcc/users/24476420516245c3bd9b57eccb5dc7ea/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:36.728191 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.728547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.728919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.756511 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] MFA Rules not processed for user `24476420516245c3bd9b57eccb5dc7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:36.757790 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.827454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1447/2863] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:36.839742 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.841640 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.899690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d936d29-afd6-426a-b6e3-51559e06a766 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.903429 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.904026 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.904301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.904571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.913415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.915504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16863a0-7596-4cb0-a798-de2402ab0e45 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.924831 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1417/2864] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:36.953613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.995225 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.015811 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e19fc2-de78-402a-80ea-9f38c5131fcb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.020482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.021107 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.021388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.021664 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] PATH_INFO: `/v3/users/9d3a89afb89d4b9f8733e14c6b6961b3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.022924 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3a89afb89d4b9f8733e14c6b6961b3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.025350 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.215527 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3a89afb89d4b9f8733e14c6b6961b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:37.227908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3a89afb89d4b9f8733e14c6b6961b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:37.234884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3a89afb89d4b9f8733e14c6b6961b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:37.235585 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3a89afb89d4b9f8733e14c6b6961b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:37.292425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3a89afb89d4b9f8733e14c6b6961b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:37.298795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1448/2865] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:36 2022] DELETE /identity/v3/users/9d3a89afb89d4b9f8733e14c6b6961b3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:37.303397 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.303682 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.303957 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.324124 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:37.325323 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:37.358760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.385969 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1418/2866] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:37.395957 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.397772 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:37.460558 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe243b0b-635c-4f5f-9045-08312532ab09 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.464272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.465121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.465399 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.465718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.474649 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.476059 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.485425 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1449/2867] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:37.916953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.918450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:37.931282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530c550-008f-4e56-8936-8f2c0a00a944 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.936155 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.938853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.938853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.938853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] PATH_INFO: `/v3/projects/fd8a6048881949f3a3c766ea6e6f4900` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.938853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8a6048881949f3a3c766ea6e6f4900)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.940502 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.075865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8a6048881949f3a3c766ea6e6f4900', 'request_id': 'req-5affae81-410e-4dea-a517-91eb8a59b176'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.166886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8a6048881949f3a3c766ea6e6f4900', 'request_id': 'req-5affae81-410e-4dea-a517-91eb8a59b176'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.173262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8a6048881949f3a3c766ea6e6f4900', 'request_id': 'req-5affae81-410e-4dea-a517-91eb8a59b176'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.185138 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] The token cache is being invalidate because project fd8a6048881949f3a3c766ea6e6f4900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:38.185424 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.186084 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.187532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1419/2868] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:37 2022] DELETE /identity/v3/projects/fd8a6048881949f3a3c766ea6e6f4900 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:38.191519 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.201054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbad536-78dc-49b4-b5db-e61aeae2d30d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.259931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.260546 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.260827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.261100 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] PATH_INFO: `/v3/users/24476420516245c3bd9b57eccb5dc7ea` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.262250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24476420516245c3bd9b57eccb5dc7ea)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.264144 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.265574 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.425964 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24476420516245c3bd9b57eccb5dc7ea'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.439363 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24476420516245c3bd9b57eccb5dc7ea'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.447318 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24476420516245c3bd9b57eccb5dc7ea'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.448034 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24476420516245c3bd9b57eccb5dc7ea'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.537444 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f8852476-780f-4277-bcfd-f1ff8a9343e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24476420516245c3bd9b57eccb5dc7ea'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:38.544245 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1450/2869] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:38 2022] DELETE /identity/v3/users/24476420516245c3bd9b57eccb5dc7ea => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:39.162914 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.164526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:39.178007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affae81-410e-4dea-a517-91eb8a59b176 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.251426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.251988 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.252253 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.252519 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] PATH_INFO: `/v3/projects/90b0da67d6514dcd9b599510195ccbcc` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.253626 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b0da67d6514dcd9b599510195ccbcc)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.255587 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.389376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b0da67d6514dcd9b599510195ccbcc', 'request_id': 'req-08e79c10-f256-469d-9d0b-32e9dd341029'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:39.469483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b0da67d6514dcd9b599510195ccbcc', 'request_id': 'req-08e79c10-f256-469d-9d0b-32e9dd341029'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:39.474454 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.475683 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b0da67d6514dcd9b599510195ccbcc', 'request_id': 'req-08e79c10-f256-469d-9d0b-32e9dd341029'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:39.486854 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] The token cache is being invalidate because project 90b0da67d6514dcd9b599510195ccbcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:39.487170 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:39.487770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08e79c10-f256-469d-9d0b-32e9dd341029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:39.489244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1420/2870] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:39 2022] DELETE /identity/v3/projects/90b0da67d6514dcd9b599510195ccbcc => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:39.494542 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.496884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.496884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.515965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:39.517182 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:39.577124 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1451/2871] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:39.584905 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.585184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.585476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.605704 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:39.606924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:39.615239 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1421/2872] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:39.619366 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.628952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39f80-0e74-4358-83df-9655a8eed4f7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.633977 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.634599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.634878 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.635148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.635786 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:39.636085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.637646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.649630 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1452/2873] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:39.655213 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.663012 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d48823-3c24-4f4f-bdc4-b78505423d3c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.667723 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.668322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.668601 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.668864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.669488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.671427 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.819511 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1422/2874] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:39 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:39.824266 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.834344 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9fefc-e06b-494f-b7e1-36323bbaeed9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.891196 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.891822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.892134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.892409 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.893052 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.894994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.907066 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.104782 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1453/2875] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:39 2022] POST /identity/v3/users => generated 291 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:40.108939 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.120057 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba119e3f-1da2-4b91-850b-3955e02ff84a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.124643 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.125240 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.125508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.125779 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.126473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.128059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.137918 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1423/2876] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:40.142261 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.152759 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a9fab-562a-43a2-9800-0c5e37f89340 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.157380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.157972 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.160788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.160788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.160788 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.161079 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b017a4-3bca-44b7-b6df-44b3804bc581 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.175324 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1454/2877] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:40.176688 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.187258 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca9e98-0b04-4d74-9e96-7626312efc8b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.191738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.192376 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.192653 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.192921 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] PATH_INFO: `/v3/projects/2796cfbec5354d6085465e77d4839b00/users/7072636de1e145b79d1a47c7690a05c0/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.207987 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2796cfbec5354d6085465e77d4839b00, user_id=7072636de1e145b79d1a47c7690a05c0, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.210666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.275875 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1424/2878] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:40 2022] PUT /identity/v3/projects/2796cfbec5354d6085465e77d4839b00/users/7072636de1e145b79d1a47c7690a05c0/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:40.344699 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.345010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.345255 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.377656 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] MFA Rules not processed for user `7072636de1e145b79d1a47c7690a05c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:40.379523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:40.458864 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1455/2879] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:40.463991 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.529970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8494c-75d2-4599-8b33-cf833bb403c9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.534630 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.535238 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.535516 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.535791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.536468 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.538365 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.654374 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1425/2880] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:40 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:40.658636 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.670118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba2267-955e-4fe2-ab4c-32934c5608ba None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.731038 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.731846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.732250 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.732637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.733387 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.735355 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.864785 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1456/2881] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:40 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:40.869344 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.880199 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab43c9-ace0-417d-9049-f370212bcbd9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.887232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.887232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.887232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.887232 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.887716 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.889081 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.901632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1426/2882] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:40.905908 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.916112 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ff6-755e-4309-813d-e149dc5b7d50 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.920774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.921423 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.921799 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.922101 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] PATH_INFO: `/v3/projects/e9cda53f116549629c9197bc65f5193e/users/eb68fc26581942999f99e8685d0fec73/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.937119 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cda53f116549629c9197bc65f5193e, user_id=eb68fc26581942999f99e8685d0fec73, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.939742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.000994 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1457/2883] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:40 2022] PUT /identity/v3/projects/e9cda53f116549629c9197bc65f5193e/users/eb68fc26581942999f99e8685d0fec73/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:41.004952 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.015393 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e1ea-a3a7-4b61-acfb-2e4ba43057de admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.071149 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.071845 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.072128 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.072405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.073047 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:41.074690 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921d3b-67cc-4923-8bef-ec8b4f5e4633 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.076800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.085016 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1427/2884] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:41.090040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.100285 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40509e4b-ca06-4064-9f5e-0f77945998a1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.104919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.105564 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.105843 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.106147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] PATH_INFO: `/v3/projects/e9cda53f116549629c9197bc65f5193e/users/eb68fc26581942999f99e8685d0fec73/roles/99f6f437de8e4cf2810a3bd858413db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.108201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cda53f116549629c9197bc65f5193e, user_id=eb68fc26581942999f99e8685d0fec73, role_id=99f6f437de8e4cf2810a3bd858413db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:41.110844 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.121161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1458/2885] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:41 2022] PUT /identity/v3/projects/e9cda53f116549629c9197bc65f5193e/users/eb68fc26581942999f99e8685d0fec73/roles/99f6f437de8e4cf2810a3bd858413db3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:41.131142 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.131423 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.131694 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.160902 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] MFA Rules not processed for user `eb68fc26581942999f99e8685d0fec73`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:41.162091 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:41.228738 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1428/2886] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:41.241064 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.243367 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:41.274345 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.304348 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fb63-f4d4-4f63-ac38-f55776b8589e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.308827 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.309442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.309940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.310755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.320033 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:41.322219 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.331907 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1459/2887] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:42.251754 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.262179 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd6b96-85f5-4b4c-abf7-12131a089321 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.266937 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb68fc26581942999f99e8685d0fec73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9cda53f116549629c9197bc65f5193e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.267765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.268024 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.268615 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.269450 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:42.269756 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.271404 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.283661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1429/2888] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.288036 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.298738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a407648-9867-49d9-8047-2ee26c0630d4 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.303975 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb68fc26581942999f99e8685d0fec73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9cda53f116549629c9197bc65f5193e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.304602 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.304886 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.305161 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.305805 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.307718 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.396866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1460/2889] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:42 2022] POST /identity/v3/projects => generated 354 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:42.401686 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.411764 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7964-9207-4552-8374-3f68a3f5b53c tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.467910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb68fc26581942999f99e8685d0fec73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9cda53f116549629c9197bc65f5193e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.468446 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.468729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.469007 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.469657 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:42.469989 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.471641 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.483907 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1430/2890] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.520613 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.531640 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf204-d00f-490a-bc1b-1d0dbeb22046 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.536246 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb68fc26581942999f99e8685d0fec73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9cda53f116549629c9197bc65f5193e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.536866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.537150 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.537426 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] PATH_INFO: `/v3/projects/1a1b086b15ee4ca7a642837256e96dd5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.538609 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1a1b086b15ee4ca7a642837256e96dd5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.540512 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.670753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1b086b15ee4ca7a642837256e96dd5', 'request_id': 'req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:42.677941 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1b086b15ee4ca7a642837256e96dd5', 'request_id': 'req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:42.694369 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1b086b15ee4ca7a642837256e96dd5', 'request_id': 'req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:42.783929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] The token cache is being invalidate because project 1a1b086b15ee4ca7a642837256e96dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:42.784262 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:42.784937 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3cd7a6e5-2007-40ba-a530-dc1b96aab0b3 tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:42.786461 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1461/2891] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:42 2022] DELETE /identity/v3/projects/1a1b086b15ee4ca7a642837256e96dd5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:43.421652 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.423463 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.487942 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbf40e-88d8-4025-b4b4-b47d7f32de3b tempest-VolumeQuotaClassesTest-1128736707 tempest-VolumeQuotaClassesTest-1128736707-project-admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.496061 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.545103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.545661 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.545941 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.546312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] PATH_INFO: `/v3/users/7072636de1e145b79d1a47c7690a05c0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.547515 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7072636de1e145b79d1a47c7690a05c0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.549435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.758795 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7072636de1e145b79d1a47c7690a05c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:43.771496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7072636de1e145b79d1a47c7690a05c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:43.779039 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7072636de1e145b79d1a47c7690a05c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:43.852770 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7072636de1e145b79d1a47c7690a05c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:43.853414 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7072636de1e145b79d1a47c7690a05c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:43.859896 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1431/2892] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:43 2022] DELETE /identity/v3/users/7072636de1e145b79d1a47c7690a05c0 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:43.864446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.864939 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.865121 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.884637 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:43.885978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.944927 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1462/2893] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:43.954967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.956452 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:44.013707 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c419a-0c12-4961-8ff0-bca8837fc6c8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.017308 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.017929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.018247 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.018518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.027971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.029408 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.038884 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1432/2894] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:44.505623 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.507201 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:44.522308 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067adbc-9354-44e6-92fe-3d40464fb059 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.527866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.528577 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.528776 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.529053 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] PATH_INFO: `/v3/projects/2796cfbec5354d6085465e77d4839b00` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.530284 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2796cfbec5354d6085465e77d4839b00)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.532272 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.596749 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.612676 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2796cfbec5354d6085465e77d4839b00', 'request_id': 'req-c5bb9fed-a941-49e6-9f13-3ce9d0300046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.619515 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2796cfbec5354d6085465e77d4839b00', 'request_id': 'req-c5bb9fed-a941-49e6-9f13-3ce9d0300046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.625028 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2796cfbec5354d6085465e77d4839b00', 'request_id': 'req-c5bb9fed-a941-49e6-9f13-3ce9d0300046'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.691474 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] The token cache is being invalidate because project 2796cfbec5354d6085465e77d4839b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:44.691730 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.692430 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.693909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1463/2895] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:44 2022] DELETE /identity/v3/projects/2796cfbec5354d6085465e77d4839b00 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:44.702028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.714044 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f897d-0aa8-4f1e-ad5b-51046d6e4866 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.745596 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.771283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.771908 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.772185 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.772460 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] PATH_INFO: `/v3/users/eb68fc26581942999f99e8685d0fec73` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.773579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb68fc26581942999f99e8685d0fec73)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.775546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.901008 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb68fc26581942999f99e8685d0fec73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.912919 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb68fc26581942999f99e8685d0fec73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.920605 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb68fc26581942999f99e8685d0fec73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.978997 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb68fc26581942999f99e8685d0fec73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.979874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-48e13697-5527-43e6-8990-d8a6b832a20e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb68fc26581942999f99e8685d0fec73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:44.987844 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1433/2896] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:44 2022] DELETE /identity/v3/users/eb68fc26581942999f99e8685d0fec73 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:45.425478 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:45.427002 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:45.440346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb9fed-a941-49e6-9f13-3ce9d0300046 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:45.500580 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:45.501142 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.501424 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.501700 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] PATH_INFO: `/v3/projects/e9cda53f116549629c9197bc65f5193e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.502859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cda53f116549629c9197bc65f5193e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:45.504745 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:45.618048 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cda53f116549629c9197bc65f5193e', 'request_id': 'req-079c894b-898b-41f9-9b76-fc2accb9ffca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:45.624410 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cda53f116549629c9197bc65f5193e', 'request_id': 'req-079c894b-898b-41f9-9b76-fc2accb9ffca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:45.629995 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cda53f116549629c9197bc65f5193e', 'request_id': 'req-079c894b-898b-41f9-9b76-fc2accb9ffca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:45.691505 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] The token cache is being invalidate because project e9cda53f116549629c9197bc65f5193e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:45.691795 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:45.692439 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-079c894b-898b-41f9-9b76-fc2accb9ffca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:45.694040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1464/2897] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:45 2022] DELETE /identity/v3/projects/e9cda53f116549629c9197bc65f5193e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:45.699632 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.699910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.700181 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.720903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:45.722125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:45.780526 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1434/2898] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:45.789415 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.789816 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.790125 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.811148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:45.812368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:45.820395 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1465/2899] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:45.825262 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:45.835862 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b6c14-4462-469c-a5cd-90f79bfba944 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:45.844560 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:45.845184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.845471 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.845791 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.846504 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:45.846809 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:45.848442 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:45.860532 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1435/2900] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:45.865054 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:45.875105 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77680-0edc-4b52-94b7-58bffb028dff None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:45.879774 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:45.880371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.880654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.880928 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.881575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:45.884168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.015654 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1466/2901] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:45 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:46.018790 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.028963 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db491-7b7c-4f33-836c-a19494fe708d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.068383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:46.086104 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.087002 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.087283 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.087558 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.088337 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.090535 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.179546 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1436/2902] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:46 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:46.186336 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.196932 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92978d3-e3a1-4317-bddf-491b4d644305 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.201575 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.202192 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.202508 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.202755 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.203421 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.204982 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.218332 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1467/2903] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:46.219607 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.230559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04632d4-94a3-4c9f-9046-a206ae6489b9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.235362 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.235972 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.236256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.236550 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.237197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.238847 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28482a0-162d-418d-985f-a5cf7fba6fff admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.250870 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1437/2904] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:46.254952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.261507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:46.265029 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb983878-64cb-4c3e-b667-2685ade8ac64 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.269686 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.270404 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.270679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.270947 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] PATH_INFO: `/v3/projects/fd222dcb984d4517b72346ee56168837/users/5e3f2492e12b4b9fb2dbe0fb8a03a9b3/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.288005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd222dcb984d4517b72346ee56168837, user_id=5e3f2492e12b4b9fb2dbe0fb8a03a9b3, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.292319 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.369589 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1468/2905] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:46 2022] PUT /identity/v3/projects/fd222dcb984d4517b72346ee56168837/users/5e3f2492e12b4b9fb2dbe0fb8a03a9b3/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:46.380559 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.380886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.381205 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.412322 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] MFA Rules not processed for user `5e3f2492e12b4b9fb2dbe0fb8a03a9b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:46.413679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:46.481954 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1438/2906] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:46.493305 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.495021 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:46.567168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f451a6-1c98-428a-866d-e01ddd9265fe admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.567168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.567168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.567168 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.567757 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.568992 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.570454 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.571974 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:46.573407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:46.579531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1469/2907] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:46.631679 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac743235-c361-4dc0-adf2-0927a45e130c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:46.635678 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:46.636360 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:46.636642 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:46.636910 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:46.638662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:46.703297 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:46.704730 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:46.708917 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:46.716123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1439/2908] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:47.408651 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:47.411716 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:47.501989 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8097525-10c3-49c0-a180-1e2867728098 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:47.509114 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:47.509724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:47.510005 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:47.510307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:47.512040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:47.524570 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:47.525490 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce1dd6-8552-4a54-8deb-801dda2590e9 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:47.535742 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1470/2909] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:50.766295 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:50.827697 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95bdd-1c06-4943-bd8e-4f608ff078a6 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:50.832212 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:50.832826 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:50.833101 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:50.833368 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] PATH_INFO: `/v3/users/5e3f2492e12b4b9fb2dbe0fb8a03a9b3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:50.834498 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3f2492e12b4b9fb2dbe0fb8a03a9b3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:50.837074 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:51.009821 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3f2492e12b4b9fb2dbe0fb8a03a9b3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.021789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3f2492e12b4b9fb2dbe0fb8a03a9b3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.028906 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3f2492e12b4b9fb2dbe0fb8a03a9b3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.091828 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3f2492e12b4b9fb2dbe0fb8a03a9b3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.092435 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3f2492e12b4b9fb2dbe0fb8a03a9b3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.098473 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1440/2910] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 11:59:50 2022] DELETE /identity/v3/users/5e3f2492e12b4b9fb2dbe0fb8a03a9b3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:51.103116 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.103618 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.103727 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.124382 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:51.125756 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:51.183812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1471/2911] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:51.193356 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:51.194787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:51.251274 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ee87-7c96-45d9-95c5-b3e29b1c2707 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:51.254885 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:51.255575 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.255864 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.256133 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.265769 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:51.267148 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed5ea7-723a-44f1-b889-082c3931ddcf service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:51.277829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1441/2912] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:51.671341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:51.676642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:51.684929 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abfa02-4acc-4715-a676-b4e0782e2035 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:51.689489 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:51.690147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.690435 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.690693 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] PATH_INFO: `/v3/projects/fd222dcb984d4517b72346ee56168837` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.691841 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd222dcb984d4517b72346ee56168837)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:51.693671 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:51.701961 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:51.806923 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd222dcb984d4517b72346ee56168837', 'request_id': 'req-3103398a-f6d6-41cd-936e-e6be4a91fb81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.813531 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd222dcb984d4517b72346ee56168837', 'request_id': 'req-3103398a-f6d6-41cd-936e-e6be4a91fb81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.819373 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd222dcb984d4517b72346ee56168837', 'request_id': 'req-3103398a-f6d6-41cd-936e-e6be4a91fb81'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.897018 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] The token cache is being invalidate because project fd222dcb984d4517b72346ee56168837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 11:59:51.897306 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.897919 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3103398a-f6d6-41cd-936e-e6be4a91fb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 11:59:51.899940 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1472/2913] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 11:59:51 2022] DELETE /identity/v3/projects/fd222dcb984d4517b72346ee56168837 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:51.904806 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.905282 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.905390 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.925184 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:51.926409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:51.939206 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:51.986666 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1442/2914] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:51.994697 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.994978 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.995268 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.014695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:52.015885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:52.024253 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1473/2915] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:52.028367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.038595 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846d683-4fab-4402-8dd5-9fe0da075cae None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.043731 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.044355 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.044635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.044903 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.045551 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:52.045852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.047500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.059014 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1443/2916] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:52.062996 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.072560 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b6a10-657d-4f82-a605-11c3dc47fb8b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.077155 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.077738 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.078010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.078311 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.078962 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.080810 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.162952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1474/2917] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Sep 9 11:59:52 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:52.167557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.177840 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64e61-4b34-4e36-8c42-eb154f7481a1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.233407 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.234011 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.234319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.234592 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.235221 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.237125 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.309010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1444/2918] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Sep 9 11:59:52 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:52.313197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.324129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87beea-3234-461b-9109-c57d582575e8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.328885 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.329482 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.329762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.330035 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.330724 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.332297 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.343185 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1475/2919] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:52.347312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.357243 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022006-4ac1-46f9-b01c-cbac0b780f14 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.361955 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.362579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.362856 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.363130 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.363777 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.365358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a834f-45be-4282-92b4-ab656d5ddde8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.376590 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1445/2920] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:59:52.380642 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.391659 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709159b-de9c-4332-9fac-7204a8e7cb40 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.397665 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.398388 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.399431 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.399663 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] PATH_INFO: `/v3/projects/6aa3f6062aa0427b823d618bf861ac3c/users/38fdb677bf0945c2bd53a1c30c16cf07/roles/d3096edb08e84cd098dda68154bffd4b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.414526 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa3f6062aa0427b823d618bf861ac3c, user_id=38fdb677bf0945c2bd53a1c30c16cf07, role_id=d3096edb08e84cd098dda68154bffd4b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.417205 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.515244 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1476/2921] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Sep 9 11:59:52 2022] PUT /identity/v3/projects/6aa3f6062aa0427b823d618bf861ac3c/users/38fdb677bf0945c2bd53a1c30c16cf07/roles/d3096edb08e84cd098dda68154bffd4b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:52.526483 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbee48-de29-4af0-8def-ddb570630b2c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.526799 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbee48-de29-4af0-8def-ddb570630b2c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.527079 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbee48-de29-4af0-8def-ddb570630b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.554481 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-99dbee48-de29-4af0-8def-ddb570630b2c None None] MFA Rules not processed for user `38fdb677bf0945c2bd53a1c30c16cf07`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:52.556187 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-99dbee48-de29-4af0-8def-ddb570630b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:52.619237 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1446/2922] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 11:59:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:52.630466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.632145 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:52.688307 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af52d3-4168-44ca-a58e-4898b32a88f3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.691953 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2eccd6d01b9a48cf946ea37543cf8c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.692544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.692822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.693094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.701870 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.703298 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.712232 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1477/2923] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:02.615912 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.708301 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.345350 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4effa-5762-46bf-a0d4-07844d378dc0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.345874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4effa-5762-46bf-a0d4-07844d378dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.345874 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4effa-5762-46bf-a0d4-07844d378dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.365579 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-06a4effa-5762-46bf-a0d4-07844d378dc0 None None] MFA Rules not processed for user `39ff00872fcf4a428520f1be1568e5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:08.366852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-06a4effa-5762-46bf-a0d4-07844d378dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:08.420850 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1447/2924] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:09.914822 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.976931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe68a3-52df-4d0b-853d-a47de208ab4e service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.982252 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.982875 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.983186 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.983460 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] PATH_INFO: `/v3/users/38fdb677bf0945c2bd53a1c30c16cf07` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.984576 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fdb677bf0945c2bd53a1c30c16cf07)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.986507 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.146944 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fdb677bf0945c2bd53a1c30c16cf07'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:10.169641 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fdb677bf0945c2bd53a1c30c16cf07'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:10.176806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fdb677bf0945c2bd53a1c30c16cf07'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:10.177633 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fdb677bf0945c2bd53a1c30c16cf07'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:10.269510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fdb677bf0945c2bd53a1c30c16cf07'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:10.276078 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1478/2925] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:00:09 2022] DELETE /identity/v3/users/38fdb677bf0945c2bd53a1c30c16cf07 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:10.280580 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.289319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.289319 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.300339 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] MFA Rules not processed for user `8ddd128e7075404193b59a38692798c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:10.301549 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.361201 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1448/2926] 10.222.0.60 () {58 vars in 958 bytes} [Fri Sep 9 12:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:10.374443 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.375935 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.434898 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96731be3-2148-4879-beb4-5dcd41bbf246 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.438859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.439965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.439965 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.440175 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.449316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.450866 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.461154 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1479/2927] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:10.860263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.861230 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.875099 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdfc5f-43e8-43cc-aecc-a5525c4b5bea None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.879807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.885269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.885269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.885269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] PATH_INFO: `/v3/projects/6aa3f6062aa0427b823d618bf861ac3c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.885269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa3f6062aa0427b823d618bf861ac3c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.885269 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.979329 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa3f6062aa0427b823d618bf861ac3c', 'request_id': 'req-c3959475-cb52-4477-8da2-858fbb23ed19'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:11.081157 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa3f6062aa0427b823d618bf861ac3c', 'request_id': 'req-c3959475-cb52-4477-8da2-858fbb23ed19'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:11.088602 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa3f6062aa0427b823d618bf861ac3c', 'request_id': 'req-c3959475-cb52-4477-8da2-858fbb23ed19'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:11.102095 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] The token cache is being invalidate because project 6aa3f6062aa0427b823d618bf861ac3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:00:11.103060 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:11.103291 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:00:11.104789 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1449/2928] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:00:10 2022] DELETE /identity/v3/projects/6aa3f6062aa0427b823d618bf861ac3c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:00:20.456380 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.098880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:39.771303 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:39.779876 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34136e-35da-4496-bef9-b120c6bff396 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:39.828565 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:39.829218 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.829498 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.829773 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.831533 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:39.903959 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:39.905372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:39.916915 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1480/2929] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:41.096263 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.097665 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:41.109967 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959475-cb52-4477-8da2-858fbb23ed19 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:41.113583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:41.114197 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.114475 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.114740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.116422 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:41.130030 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:41.131476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:41.142936 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1450/2930] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:49.911374 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=54 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:51.138103 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:59.741147 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:59.750997 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3264737-9564-4039-b74e-7e1cae7a0861 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:59.754691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:59.755274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.758909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.758909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.758909 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:59.821213 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:59.822859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:59.832767 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1481/2931] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:03.194367 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.195971 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:03.267488 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e02ae-cdfc-4be0-a341-cbf2e911fda6 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:03.272570 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:03.273175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.273500 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.273740 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.276032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:03.287824 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:03.293492 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:03.299752 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1451/2932] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:07.235820 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.237346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:07.252784 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c9cf-cff5-447e-a3de-fa418636b92f service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.256510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.257173 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.257476 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.257762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] PATH_INFO: `/v3/users/1c081d9bf3744177a2d2d50962a38027` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.272197 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c081d9bf3744177a2d2d50962a38027)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.274159 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:07.505368 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c081d9bf3744177a2d2d50962a38027'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:07.519455 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c081d9bf3744177a2d2d50962a38027'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:07.527087 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c081d9bf3744177a2d2d50962a38027'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:07.527797 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c081d9bf3744177a2d2d50962a38027'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:07.599680 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c081d9bf3744177a2d2d50962a38027'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:07.606599 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1482/2933] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:01:07 2022] DELETE /identity/v3/users/1c081d9bf3744177a2d2d50962a38027 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:07.858523 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.871829 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806f71-58f8-4972-bf06-686887c886fb service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.940165 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.940800 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.941086 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.941359 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] PATH_INFO: `/v3/projects/3e08177037d74900a886e0aa2ae057ae` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.952573 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e08177037d74900a886e0aa2ae057ae)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.954524 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:08.069760 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e08177037d74900a886e0aa2ae057ae', 'request_id': 'req-15e47ada-82b2-4f77-aaa9-80b5722f63e2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.188563 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e08177037d74900a886e0aa2ae057ae', 'request_id': 'req-15e47ada-82b2-4f77-aaa9-80b5722f63e2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.194606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e08177037d74900a886e0aa2ae057ae', 'request_id': 'req-15e47ada-82b2-4f77-aaa9-80b5722f63e2'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.206886 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] The token cache is being invalidate because project 3e08177037d74900a886e0aa2ae057ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:01:08.207175 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.207808 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.209383 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1452/2934] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:01:07 2022] DELETE /identity/v3/projects/3e08177037d74900a886e0aa2ae057ae => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:08.213544 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:08.214970 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:08.228608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93552319-a0e1-4ac2-9549-ac4dc492dc52 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:08.278447 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:08.285151 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:08.292901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.292901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.292901 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] PATH_INFO: `/v3/users/c03f54407e3c456cb42c55bc67dd61cf` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.301739 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03f54407e3c456cb42c55bc67dd61cf)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:08.304118 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:08.487930 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03f54407e3c456cb42c55bc67dd61cf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.500510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03f54407e3c456cb42c55bc67dd61cf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.507293 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03f54407e3c456cb42c55bc67dd61cf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.507954 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03f54407e3c456cb42c55bc67dd61cf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.570900 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03f54407e3c456cb42c55bc67dd61cf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:08.576945 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1483/2935] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:01:08 2022] DELETE /identity/v3/users/c03f54407e3c456cb42c55bc67dd61cf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:08.783985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:08.785518 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:08.799028 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47ada-82b2-4f77-aaa9-80b5722f63e2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:08.807925 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:08.856343 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:08.856992 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.857276 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.857571 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] PATH_INFO: `/v3/projects/0d80463eaafe40c7b583921eb79b4d31` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.869288 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d80463eaafe40c7b583921eb79b4d31)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:08.871415 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:08.989272 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d80463eaafe40c7b583921eb79b4d31', 'request_id': 'req-294e0c80-ca12-43cf-97ce-08e64ce9856d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:09.057476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d80463eaafe40c7b583921eb79b4d31', 'request_id': 'req-294e0c80-ca12-43cf-97ce-08e64ce9856d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:09.063583 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d80463eaafe40c7b583921eb79b4d31', 'request_id': 'req-294e0c80-ca12-43cf-97ce-08e64ce9856d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:09.076377 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] The token cache is being invalidate because project 0d80463eaafe40c7b583921eb79b4d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:01:09.076662 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:09.077301 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:09.078881 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1453/2936] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:01:08 2022] DELETE /identity/v3/projects/0d80463eaafe40c7b583921eb79b4d31 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:01:18.573417 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:19.073671 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:25.669215 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:25.670711 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:25.683804 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cda6d3-e1c1-44cf-bcdd-8b6a681f9321 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:25.741084 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:25.741753 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:25.742026 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:25.742325 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] PATH_INFO: `/v3/users/39ff00872fcf4a428520f1be1568e5c6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:25.743427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ff00872fcf4a428520f1be1568e5c6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:25.745316 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:25.969047 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ff00872fcf4a428520f1be1568e5c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:25.981076 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ff00872fcf4a428520f1be1568e5c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:25.987969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ff00872fcf4a428520f1be1568e5c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:25.988627 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ff00872fcf4a428520f1be1568e5c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.085274 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ff00872fcf4a428520f1be1568e5c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.092270 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1484/2937] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:01:25 2022] DELETE /identity/v3/users/39ff00872fcf4a428520f1be1568e5c6 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:26.328159 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:26.329801 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:26.344765 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e0c80-ca12-43cf-97ce-08e64ce9856d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:26.412332 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:26.412970 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:26.413242 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:26.413508 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] PATH_INFO: `/v3/projects/420ff8c80d8542c7818d2c7caafb4eb7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:26.414651 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420ff8c80d8542c7818d2c7caafb4eb7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:26.417110 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:26.520135 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420ff8c80d8542c7818d2c7caafb4eb7', 'request_id': 'req-5dfc09ad-48b7-4710-a536-c3e835202526'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.570928 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420ff8c80d8542c7818d2c7caafb4eb7', 'request_id': 'req-5dfc09ad-48b7-4710-a536-c3e835202526'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.576855 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420ff8c80d8542c7818d2c7caafb4eb7', 'request_id': 'req-5dfc09ad-48b7-4710-a536-c3e835202526'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.588505 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] The token cache is being invalidate because project 420ff8c80d8542c7818d2c7caafb4eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:01:26.588793 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.589426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:01:26.591198 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1454/2938] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:01:26 2022] DELETE /identity/v3/projects/420ff8c80d8542c7818d2c7caafb4eb7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:36.088110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:36.585426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:00.015160 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:00.016547 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:00.072662 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7db7-91a4-442b-be90-2268a43e11d4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:00.121278 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:00.121884 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:00.122206 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:00.122484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:00.124211 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:00.134134 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:00.135545 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:00.143717 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1485/2939] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:10.140031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=46 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:45.522696 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:45.524174 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:45.537214 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc09ad-48b7-4710-a536-c3e835202526 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:45.540807 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:45.541403 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:45.541684 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:45.541958 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:45.543687 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:45.554119 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:45.555496 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:45.563476 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1455/2940] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:45.994031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:45.995470 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:46.006497 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521d6ee-d295-402d-b751-1a2d1a2915d0 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:46.010442 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:46.011040 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:46.011337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:46.011604 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:46.013466 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:46.023646 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:46.025061 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:46.033523 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1486/2941] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:46.107059 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:46.117208 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724213-ed29-421e-9187-b941730cbe01 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:46.120985 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:46.121510 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:46.121784 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:46.122063 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:46.123744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:46.198861 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:46.200316 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:46.211388 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1456/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:46.271952 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:46.273372 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:46.351955 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a5abb-19ce-47f5-aafb-024e7de3c780 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:46.356922 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:46.357524 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:46.357806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:46.358106 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:46.359910 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:46.372695 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:46.374281 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:46.384371 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1487/2943] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:56.206357 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=40 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:56.379969 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=38 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:31.026880 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:31.062812 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c0d98-d139-4127-92ce-a83e99fd0d66 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:31.067010 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:31.067787 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:31.068351 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:31.068843 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:31.071409 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:31.154663 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:31.156107 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:31.165766 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1457/2944] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:04:32.066446 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:32.068346 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:32.157812 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9df31-3dbe-4545-b069-834061da3d18 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:32.163485 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:32.164110 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:32.164385 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:32.164657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:32.166549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:32.180085 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:32.181736 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:32.193691 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1488/2945] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:34.822712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:34.835805 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9607c-9894-4fdb-b620-7fdd3df1b85b service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:34.841522 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7b89441837347cba3200f9104c0f16c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:34.845254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:34.845254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:34.845254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:34.845254 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:34.857248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:34.858853 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:34.869354 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1458/2946] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:37.552337 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:37.561934 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b523-c8c5-4d70-8310-0b57fca7fd0d service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:37.565510 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b062fa227cd043719c7eff9d2b7e4bf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:37.566128 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:37.566408 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:37.566679 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:37.568398 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:37.652552 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:37.653960 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:37.672857 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1489/2947] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:44.701620 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:44.703248 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:44.742849 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b44fe0-8117-4083-883e-91bf02bc27b7 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:44.747852 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:44.748489 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:44.748763 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:44.749040 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:44.750751 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:44.762889 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:44.764557 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:44.775341 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1459/2948] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:44.814541 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:44.816073 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:44.829042 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c9b76-776a-49f8-9891-4ab82f308ed4 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:44.833712 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:44.834300 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:44.834581 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:44.834852 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] PATH_INFO: `/v3/users/f5a780d912d64a7f9b8238f8021f7b24` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:44.836491 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a780d912d64a7f9b8238f8021f7b24)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:44.838419 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:44.991148 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a780d912d64a7f9b8238f8021f7b24'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:44.995459 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:45.003286 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a780d912d64a7f9b8238f8021f7b24'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.010427 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a780d912d64a7f9b8238f8021f7b24'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.011094 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a780d912d64a7f9b8238f8021f7b24'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.087004 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a780d912d64a7f9b8238f8021f7b24'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.093384 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1490/2949] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:04:44 2022] DELETE /identity/v3/users/f5a780d912d64a7f9b8238f8021f7b24 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:04:45.350077 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:45.361943 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a066fd-e0bd-4ef1-820b-2187a6c02aa5 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:45.424078 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:45.424701 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:45.424980 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:45.425250 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] PATH_INFO: `/v3/projects/d19dac54b1ce4571b533178a1dbc4c39` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:45.426405 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19dac54b1ce4571b533178a1dbc4c39)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:45.428280 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:45.435173 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:45.549453 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19dac54b1ce4571b533178a1dbc4c39', 'request_id': 'req-1209d46a-151a-44e4-a86f-9bcda6581546'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.635004 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19dac54b1ce4571b533178a1dbc4c39', 'request_id': 'req-1209d46a-151a-44e4-a86f-9bcda6581546'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.640929 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19dac54b1ce4571b533178a1dbc4c39', 'request_id': 'req-1209d46a-151a-44e4-a86f-9bcda6581546'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.652426 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] The token cache is being invalidate because project d19dac54b1ce4571b533178a1dbc4c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:04:45.652746 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.653398 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:45.654924 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1460/2950] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:04:45 2022] DELETE /identity/v3/projects/d19dac54b1ce4571b533178a1dbc4c39 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:04:45.659010 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:45.660608 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:45.673918 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d9858-3e6b-49f7-8c7c-7a55468c63eb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:45.730806 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:45.731433 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:45.731720 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:45.731980 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] PATH_INFO: `/v3/users/de9f32a80a25484d90856dce4f4d6114` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:45.733183 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9f32a80a25484d90856dce4f4d6114)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:45.735129 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:45.991264 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9f32a80a25484d90856dce4f4d6114'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.003057 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9f32a80a25484d90856dce4f4d6114'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.009859 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9f32a80a25484d90856dce4f4d6114'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.010549 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9f32a80a25484d90856dce4f4d6114'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.080341 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9f32a80a25484d90856dce4f4d6114'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.086764 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1491/2951] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Sep 9 12:04:45 2022] DELETE /identity/v3/users/de9f32a80a25484d90856dce4f4d6114 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:04:46.340635 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:46.342334 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:46.357865 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209d46a-151a-44e4-a86f-9bcda6581546 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:46.422032 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddd128e7075404193b59a38692798c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e4bd719a3a8247ec9289af5d26132989', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:46.422675 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:46.422953 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:46.423229 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] PATH_INFO: `/v3/projects/570155e09fd5431384350cd10f5f8ec8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:46.424333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570155e09fd5431384350cd10f5f8ec8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:46.426256 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:46.554333 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570155e09fd5431384350cd10f5f8ec8', 'request_id': 'req-24c7e668-e2c9-45a2-b5f3-b29cdf253558'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.635191 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570155e09fd5431384350cd10f5f8ec8', 'request_id': 'req-24c7e668-e2c9-45a2-b5f3-b29cdf253558'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.641490 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570155e09fd5431384350cd10f5f8ec8', 'request_id': 'req-24c7e668-e2c9-45a2-b5f3-b29cdf253558'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.653054 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] The token cache is being invalidate because project 570155e09fd5431384350cd10f5f8ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:04:46.653358 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.654001 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:04:46.655501 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1461/2952] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Sep 9 12:04:46 2022] DELETE /identity/v3/projects/570155e09fd5431384350cd10f5f8ec8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:04:49.347517 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:49.348931 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:49.404070 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:49.424657 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824b062-1338-410b-b478-cc38f4ea73e9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:49.485846 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:49.486484 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:49.486762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:49.487031 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:49.488762 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:49.501074 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:49.502848 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebf522-a375-40ab-9d36-898d267a841a service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:49.512867 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1492/2953] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:56.649890 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:59.514842 dv-cinder-856460-2-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=36 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:53.287178 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:53.288606 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:53.304244 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7e668-e2c9-45a2-b5f3-b29cdf253558 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:53.309123 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0c95e5281f4f9d9dac910415d10932', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e972aecf7fdf4696814c61481ddcca9e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:53.309712 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:53.309990 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:53.310293 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:53.318744 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:53.320312 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27faa77-67af-455a-9fc8-631bb3da0242 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:53.330729 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1462/2954] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:10:03.326433 dv-cinder-856460-2-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}